KVM Virtualization 201

 

Background

In my original post KVM Virtualization 101 and my second post KVM Virtualization 102, I explained some basic commands that are necessary to get on using KVM and some features for tuning KVM. In this article I move on to some of the more advanced problems when converting existing Linux machines from a physical machine to a virtual machine. This is called P2V conversion.

When converting, there are a couple of things that often change. I will outline the major pitfalls below. These techniques have been used to successfully convert RHEL4, RHEL5 and even ES/AS 2.1 machines.

 

Lab Tools

Rescue Disk

Personally, I use two main rescue disks, the Red Hat rescue disk and Puppy Linux, but any live CD will do. The Gentoo live CD/DVD and Knoppix are also very good.

Just get one you are comfortable with and use it, but always have a rescue disk on hand from the Linux distribution for which you are working on, especially if it is older. When I work on a Red Hat system, I always use a Red Hat rescue CD because the kernel versions, file system utilities, etc will be the same version as what is on the file system and everything works together.

The main reason to fall back to a different Live CD/DVD than your Linux vendor is when you need other network utilities or security tools, not when converting virtual machines.

 

Change Root (chroot)

This command is critical to have access to when you are converting virtual machines by hand and is available on almost any live CD ever produced. It usually works best with your vendor specific disk. Many of the live CD/DVDs help you by automatically mounting /dev, /sys, and /proc. Others, you will have to do this manually, so your millage may vary

The Gentoo Manual provides an excellent primer on using chroot manually

 

Grub Installer (grub-install)

Grub comes with an installer utility and you will need to use it. The installer utility writes the stage1 file to the MBR of the disk for which you are working on. This is true in virtual machines and host machines. Grub installer automatically detects some parameters for you based on your fstab, devices.map file, and grub.conf file.

 

Common Pitfalls

The following commands are all ran from within the chroot environment

FIle System Table (/etc/fstab)

When converting a machine, the first thing to look at is the File System Table file also called fstab. As of his writing KVM only supports ATA devices which are also are identified as /dev/hda or /dev/hdb in Linux.

First, always back up the current file

cp /etc/fstab /etc/fstab.bak

Then change the fstab to match the new KVM devices. Notice, we are converting from a Compaq Raid Array

/dev/cciss/c0d0p1 / ext3 defaults 1 1
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0

Change to:
/dev/hda2 / ext3 defaults 1 1
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0

 

Grub Device Map (/boot/grub/device.map)

Again, backup the file, then change from the Compaq Raid array to the new ATA device. This file is what maps the grub name to the Linux Kernel name.

(fd0) /dev/fd0
(hd0) /dev/cciss/c0d0

Chang to
(fd0) /dev/fd0
(hd0) /dev/hda

 

Grub Configuration File (/boot/grub/grub.conf)

There are several caveats to the grub configuration file below. In our example, we are moving from a Linux system which had swap & boot located on separate raid volumes, so our root file system was located on what grub considered device 0, partition 0. With the new KVM disk, it was easier to specify the Swap partition as 2048MB and let the /root filesystem have the rest of the space. This has the byproduct of forcing us to change the grub.conf file

The grub device (hd0,0) becomes (hd0,1). Also, notice that since we had a separate boot partition that the kernel and initrd image were both addressed from /, this changes in our virtual environment. Since everything will now reside together on the same partition, we must readdress the path to begin with /boot.

Was:
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/cciss/c0d0p2
# initrd /initrd-version.img
#boot=/dev/cciss/c0d0
default=1
timeout=5
#splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux ES (2.6.9-22.0.2.EL)
root (hd0,0)
kernel /vmlinuz-2.6.9-22.0.2.EL ro root=/dev/cciss/c0d0p2 quiet
initrd /initrd-2.6.9-22.0.2.EL.img

Change to:

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,1)
# kernel /vmlinuz-version ro root=/dev/hda2
# initrd /initrd-version.img
#boot=/dev/hda
default=1
timeout=5
#splashimage=(hd0,1)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux ES (2.6.9-22.0.2.EL)
root (hd0,1)
kernel /boot//vmlinuz-2.6.9-22.0.2.EL ro root=/dev/hda2 quiet
initrd /boot//initrd-2.6.9-22.0.2.EL.img

 

Install Grub Stage 1 (grub, grub-install)

Grub can be installed manually with the grub command or automatically with the grub-install command.

To install grub automatically for the above configuration, use the command below. Notice the disk which is specified. Note: this must be done from inside of a chroot environment. Also note that this is the disk name and not the partition because we want the stage1 installed in the MBR.

This is one of the cases where using the vendor supplied rescue disk can come in handy. I tried to use Puppy Linux while converting a RHEL 4 box and realized that puppy linux recognized ATA drives as /dev/sdx, while Red Hat recognizes them as /dev/hdx. You can also work around this by changing fstab & grub.conf temporarily

grub-install /dev/hda

To install manually for the above configuration, use the following command

grub
root (hd0,1)
setup (hd)
quit

The Gentoo Manual also provides an excellent primer on installing grub’s stage1 file.

 

Other Caveats

Sometimes directories like /tmp are not backed up, so you might need to manually create some of them when using a backup copy as the basis of virtual machine conversion. The temp directory /tmp can be tricky because of special permissions.

Create /tmp directory

mkdir /tmp
chmod a=rwxt /tmp