Another root filesystem recovery HOWTO.
Recently I had a hard disk develop major faults such that the root filesystem went readonly. Although rebooting caused it to come back up fine, a SMART check (
Since they could not fit both the old and new disks in the server at once, instead they copied the contents of the failing disk to a
Rather than try to upgrade the new installation to Fedora Core 6 and get everything configured back to the way it was, instead I opted to swap the files in
If you try to do "
Instead, I used
The only thing that went wrong with this approach was that rebooting failed, I had to ask the server operators to manually restart the machine. I think this was probably due to a problem with the shutdown sequence.
If you have problems with files / resources being "busy", make sure you have stopped all non-essential services - i.e. anything other than the network interfaces and SSH - before you start the move. Also, note you won't be able to move
Recently I had a hard disk develop major faults such that the root filesystem went readonly. Although rebooting caused it to come back up fine, a SMART check (
hdparm -t long /dev/hda)
showed that it was failing, so I requested that the server operators
replace the hard disk (it's a leased server not under my direct
control).Since they could not fit both the old and new disks in the server at once, instead they copied the contents of the failing disk to a
/backup directory on the new disk, which they had installed a fresh copy of Fedora Core 4 on to.Rather than try to upgrade the new installation to Fedora Core 6 and get everything configured back to the way it was, instead I opted to swap the files in
/backup/ with the current /, i.e.
replace the whole new root filesystem with the old one to get
everything back as it was before the change. I also had to do this live
over the network, since I have no physical access to the server.If you try to do "
mv /bin /OLD/bin; mv /backup/bin /bin" you will run into major problems - after the first mv, the mv
command won't work because it has moved to a different directory, and
if you work around that, you will still eventually run into nasty
problems related to /lib.Instead, I used
mount --bind / /backup/mnt to make the root filesystem visible under /backup so that I could then do chroot /backup (making sure /backup/dev/ was populated with device files first). I was then able to replace, under /backup, /mnt/{bin,boot,etc,lib,...} to replace the "real" root filesystem's files (i.e. mv /mnt/bin /mnt/old-bin; cp -a /bin /mnt/bin etc).The only thing that went wrong with this approach was that rebooting failed, I had to ask the server operators to manually restart the machine. I think this was probably due to a problem with the shutdown sequence.
If you have problems with files / resources being "busy", make sure you have stopped all non-essential services - i.e. anything other than the network interfaces and SSH - before you start the move. Also, note you won't be able to move
/dev, /proc, /sys, or any other directories that are mount points or which have mount points under them (though you could investigate mount --move).
