View source for Migrate CentOS 5.4 to Ovirt

==Purpose==
A very old install of CentOS 5.4 needed to be migrated to an Ovirt Hypervisor. This was a special install of Hylafax, a productions server that could not be taken offline, and was critical to business operations.
<br>
<br>
Several attempts were made to replicate the server and it's 500GB of data. Including running a disk dump over ssh to the destination VM and another physical server.
==Steps==
My final idea was to rsync the system to the new VM disk, install grub, then go from there.
===Rsync===
First I created a VM in Ovirt with a disk large enough to hold the data, plus extra as the existing server was running out of space.
<br>
After that was completed I
*booted off a live fedora ISO
*ran fdisk
**created first partition with enough space for the boot partition.
**created second partition with rest of space.
*Formatted both partitions as ext3, same as the old server.
*started sshd
*created a password for liveuser
*On the old server, ran ssh-copy-id liveuser@newvm to ensure key based ssh was setup.
====rsync boot partition====
While still in the live fedora shell
*mount /dev/sda1 on /mnt
*chown liveuser /mnt
Then on the old server ran the following dry run to check for errors.
<pre>
rsync -avn /boot/* liveuser@newvm:/mnt/ &> rsync.log
</pre>
Then the live transfer
<pre>
rsync -avz /boot/* liveuser@newvm:/mnt/ &> rsync.log
</pre>
====rsync root partition====
Back in the line fedora shell
*umount the last partition used for the boot partition on /mnt
*mount the other partition created (mount /dev/sda2 /mnt)
*chown liveuser /mnt (shouldn't be needed, just in case).
Then on the old server I ran the following dry run to check for errors and loops. Making sure to exclude the boot partition we already replicated.
<pre>
rsync -avn --exclude=\/boot / liveuser@newvm:/mnt/ &> rsynctest2.log &
disown
</pre>
*Every so often I would check to make sure this was still running. Again, this was 500GB of small files on old mechanical disks.
*The next morning things looked good, so we went for a live migration. This time adding a nice value of +10 to ensure the replication did not impact system performance during production.
<pre>
nice rsync -avz --exclude=\/boot / liveuser@newvm:/mnt/ &> rsynctest.log &
disown
</pre>
*After a day or so this finished without any strange errors, beyond the expected "files disappeared" that you would get from replicating a live system.
===Installing and fixing grub===
This was the part that took the most time. For this part I
*Booted the VM from a CentOS 7 minimal install ISO.
*Jumped to the rescue shell. At this point it did not find any linux volumes, which was something I had seen after performing the disk dump.
*Ran fdsik to verify the partitions, unlike the disk dump I was able to see the two volumes I had created previously. PROGRESS!
====Install Grub====
I ran into an issue with the chroot environment not recognizing the new /dev/sda disk that the VM uses. This was due to the old server being installed on an HP with smart raid controller<ref>https://wiki.centos.org/TipsAndTricks/ReinstallGRUB</ref>. The original disk would show up as /dev/cciss/c0d0.
<br>
<br>
To get around this I copied the grub config files to the live environment that did see the disk (this was before I realized a process that would have fixed this for me)
*mounted both partitions as follows
**mount /dev/sda2 /mnt/sysimage
**mount /dev/sda1 /mnt/sysimage/boot
*Copied /mnt/sysimage/usr/share/grub/x86_64-redhat to /usr/share/grub/x86_64-redhat because the rescue environment did not have this.
*Updated grub.conf with the new drive information (switching from c0d0 to sda)
*Installed grub onto the new disk <ref>https://wiki.centos.org/HowTos/GrubInstallation</ref>
<pre>
grub-install /dev/sda
</pre>
SUCCESS! We were able to boot grub, but then kernel panics as it was unable to find the root partition.
====Fixing Grub====
The first thing I did was to rebuild initrd with virtio modules. Thankfully the old server already had virtio modules available.<ref>https://wiki.centos.org/TipsAndTricks/CreateNewInitrd</ref>
*Booted back into the CentOS 7 rescue environment.
*Did the following to properly chroot (this would have been helpful in the last step)
<pre>
mount --bind /proc /mnt/sysimage/proc
mount --bind /dev /mnt/sysimage/dev
mount --bind /sys /mnt/sysimage/sys
chroot /mnt/sysimage
</pre>
*At this point I checked for the kernel version I needed
<pre>
ls -1 /boot/initrd-*
/boot/initrd-2.6.18-164.9.1.el5.img
/boot/initrd-old
/boot/initrd-old-
</pre>
*Using 2.6.18-164.9.1.el5, I generated a new initrd image with virtio moduels <ref>https://www.centos.org/forums/viewtopic.php?t=14647</ref> (after making a backup of course)
<pre>
mkinitrd --with=virtio_pci --with=virtio_blk -f /boot/initrd-2.6.18-164.9.1.el5.img
</pre>
At this point I was still getting a kernel panic with root mount errors, though I did see the virtio modules being loaded. Everything I had read online said to NOT use UUIDs in grub, but since I was out of ideas it was the next thing to try.
<br>
<br>
Back in the CentOS 7 rescue shell, and chroot back into the CentOS 5.4
<pre>
uuid=$(ls -l /dev/disk/by-uuid | grep sda2 | awk '{print $9}')
sed -i "s|/dev/sda2|UUID=$uuid|" /boot/grub/grub.conf
sed -i "s|/dev/sda2|UUID=$uuid|" /etc/fstab
</pre>
*Another reboot, and I had a login prompt!
<br>
<br>