Development/Howto/Xen

From Mandriva

Jump to: navigation, search
How to setup and use Xen on Mandriva Linux

Contents


[edit]

Introduction

Xen is a virtual machine monitor for x86 hardware (runs on i686 and x86-64 CPUs class), which supports running multiple guests operating systems on a single machine. Guest OS (also called "domains") require a modified kernel which supports Xen hypercalls in replacement to access to the physical hardware. At boot, the Xen kernel is loaded (via grub) along with the guest kernel for the first domain (called domain0). domain0 has privileges to access the physical hardware (PCI and ISA devices), administrate other domains and provide virtual devices (disks and network) to other domains. For more details, see www.cl.cam.ac.uk.

For this example, I'm using an old Pentium III 550Mhz with 256MB Ram, and a 12 Gig Hard drive. During the installation, I've created a partition of 5 Gigabytes mounted on '/' for the domain0 installation. And additionally a swap partition of 256MB mounted on /dev/hda5.

With 7 Gigabytes left, I create two pratitions of 3.2 Gigabytes not mounted for two guest domains os installations. And two swaps of 256MB.

My partition table:

root domain0 -> hda1
swap domain0 -> hda5

root guest1 -> hda6
swap guest1 -> hda7

root guest2 -> hda8
swap guest2 -> hda9
[edit]

Installing and configuring domains

[edit]

Installing domain0

The following is a list of requisites packages :

  • xen: The Xen kernel itself and tools for Xen system. Tools permit virtual machines monitoring and management.
  • kernel-xen0: The Xen domain0 Linux kernel. The "0" suffixed privileged versions is used to boot the system.
  • kernel-xenU: The Xen guest domain Linux kernel. The "U" suffixed unprivileged versions is used to boot guests domains.

Full rpms depencies list:

  • kernel-xen0-2.6.12.12mdk-2.6.12-12mdk.i586
  • kernel-xenU-2.6.12.12mdk-2.6.12-12mdk.i586
  • xen-3.0-0.20050823.6mdk.i586 =>
    • fontconfig-2.3.2-5mdk.i586
    • glibc-xen-2.3.5-5mdk.i586
    • iptables-1.3.3-3mdk.i586
    • libSDL1.2-1.2.8-5mdk.i586
    • libexpat0-1.95.8-1mdk.i586
    • libfontconfig1-2.3.2-5mdk.i586
    • libfreetype6-2.1.10-8mdk.i586
    • libgmp3-4.1.4-1mdk.i586
    • libltdl3-1.5.18-1mdk.i586
    • libnas2-1.7b-1mdk.i586
    • libxorg-x11-6.9-1.cvs20050915.2mdk.i586
    • libxpm4-3.4k-31mdk.i586
    • pycrypto-2.0-1mdk.i586
    • python-2.4.1-3mdk.i586
    • python-OpenSSL-0.6-2mdk.i586
    • python-base-2.4.1-3mdk.i586
    • python-twisted-core-2.0.1-1mdk.i586
    • python-zope-interface-3.0.1-1mdk.i586
    • iproute2-2.6.10-2mdk.i586
    • bridge-utils-1.0.6-1mdk.i586
    • zlib1-devel-1.2.3-1mdk.i586
[edit]

Configuring domain0

[edit]

GRUB configuration

An entry must be added to the grub configuration file menu.lst (in Mandriva Linux 2006, it's located in /boot/grub) so Xen / Xenolinux can boot. The entry should look like this:

title XEN 3 / Mandriva 2006.0
kernel (hd0,0)/boot/xen.gz dom0_mem=131072
module (hd0,0)/boot/vmlinuz-2.6.12-18mdkxen0 root=/dev/hda1 ro
module (hd0,0)/boot/initrd-2.6.12-18mdkxen0.img

kernel line tells grub where Xen itself is located and parameters passed to the kernel (in this case, setting the domain0 size of allocation memory in kilobytes).

module first line tells where to find Xenolinux kernel that Xen should launch and the parameters it should be passed. These are standard Linux parameters, root device identification and inital read-only mount.

module second line tells the path of the initrd. It must be module and not initrd in the grub configuration, else Xen doesn't boot.

Create initrd as follows:

# mkinitrd -v -f /boot/initrd-2.6.12-18mdkxen0.img 2.6.12-18mdkxen0

When adding a new kernel in menu.lst, it is recommended to not delete existing entries because you could have to reboot on your old Linux kernel if you have problems.

[edit]

TLS libraries

With Mandriva Linux 2006 kernel-xen0 and generaly all Xenolinux2.6 kernels, it is advised to disable the Thread Local Storage (TLS) library beforethe first Xenolinux boot. You can disable TLS with the following command:

# mv /lib/tls /lib/tls.disabled

You can restore TLS in the original directory:

# mv /lib/tls.disabled /lib/tls

The reason for this is that the current TLS implementation uses segmentation that is not permissible under Xen. If TLS is not disabled, an emulation mode is used in Xen which reduces performance substantially. To ensure full performance you should install a `Xen-friendly' version of the library.

[edit]

Starting Xen services

After the end of the installation and configuration, reboot and choose the Xen entry in the grub menu.

During the boot, the first part of the output display is from Xen itself, with low level information on itself and hardware. The last part of the output display is by *XenoLinux*.

First, for creating additionnal domains, we should start xend daemon control. We could start the xendomains daemon too, which launches additonnal domains at domain0 boot.

# chkconfig --add xend
# chkconfig --add xendomains
# service xend start
# service xendomains start

At this point, you could use the xm tool to monitor or maintain the domains running on your system.

[edit]

Guest domain installation

The first step in creating a new additional domain is to prepare a root filesystem for it to boot. Typically, this might be stored in a normal partition, an LVM partition, a disk file or on an NFS server. A simple solution to do this is to boot from the Mandriva Linux 2006 installation disk and install the distribution into a new partition of you hard drive.

In the following sections, I give another solutions based on a copy of the domain0 root partition, installed with urpmi in a physical partition or in a file.

We will use $mymnt to keep it simple, for mount point:

# export mymnt=/mnt/xen
# mkdir -p $mymnt
[edit]

Copy partition

The main avantage of this technique is that an installation disk is not needed. We copy the root partition of domain0 to another partition of the hard drive to obtain a second filesystem where guest domain is used.

# mount /dev/hda6 $mymnt
# rsync -avDx / $mymnt
# cp -ar /dev/* $mymnt/dev/
# umount $mymnt

Warning! The rsync command only treats the root filesystem, it does not follow mounts linked to other filesystems. If you have separate /usr and /var partitions, redo it for each mount point, except network and special entries. You want only the system as separate:

You have /usr and /var partitions:

for i in "var usr"; do
  mkdir ${mymnt}/${i}
  rsync -avDx /${i} $mymnt/${i}
done

If you have different partition schemes between domain0 and additionnal domain, do not forget to adapt the /etc/fstab file to the guest domain filesystem.

[edit]

Install Mandriva 2006.0 with urpmi

With an urpmi installation, you obtain a clean install of a new Mandriva Linux 2006.

  • First, mount your clean partition:
# mount /dev/hda6 $mymnt
  • Make an rpm repository:
# mkdir -p $mymnt/var/lib/rpm
# rpm --initdb --root $mymnt
  • Set up urpmi media:

Set up urpmi sources using your favorite mirror. The website club.mandriva.com has a nice tool for selecting mirrors and easily setting up urpmi sources. You have three combo box where you select your architecture, your distribution version and your resource type then click on Search corresponding mirrors. A list of mirrors appears and you can select your more localized mirror. Click on the mirror's URPMI set up link and type (or copy/paste) the urpmi command which is like:

# urpmi.addmedia "2006_distrib" \
   ftp://ftp.lip6.fr/pub/linux/distributions/Mandrakelinux/official/2006.0/i586/media/main/ \
   with ./media_info/hdlist.cz

Now urpmi is set up, you can install all packages you want.

  • Install guest domain Mandriva Linux 2006 system base and urpmi for installing new packages.
# urpmi --root $mymnt basesystem urpmi

Prompt will ask you many questions. Answer there just as you would make a normal installation.

The following steps configure the last features:

  • Copy config files from domain0.
# cp /etc/resolv.conf $mymnt/etc
# cp /etc/fstab $mymnt/etc
  • Copy networking configuration from domain0:
# cp /etc/sysconfig/network-scripts/ifcfg-eth0 $mymnt/etc/sysconfig/network-scripts
  • Installing dhcp client if necessary:
# urpmi --root $mymnt dhcp-client
  • Enter chroot for final configuration
# chroot $mymnt
  • Create empty necessary config files:
# touch /etc/mtab /etc/urpmi/urpmi.cfg /var/lib/urpmi/MD5SUM
  • Create shadow from passwd:
# pwconv
  • Disable tls libraries:
# mv /lib/tls /lib/tls.disabled
# ldconfig
  • Turn on the network by creating the /etc/sysconfig/network file with following content:
NETWORKING=yes
  • Set root password:
# passwd root
  • Exit chroot's shell.
# exit
  • Umount the guest partition:
# umount $mymnt
[edit]

Install Mandriva Linux 2006 in a file

This technique has as main advantage that it does not modifiy the partition structure of your hard drive. I will explain how to create and mount a disk image; then you can choose one of two methods below to install Mandriva Linux 2006 (urpmi or copy of the root partition).

  • First, create an image file for your Mandriva Linux 2006 guest domain. This command creates a disk image of 1GB, fill of zero, in a file named mandriva.img located in the current directory.
# dd if=/dev/zero of=mandriva.img bs=1M count=1 seek=1024
  • Now, create a filesystem in the image file. The -j specifies an ext3 filesystem.
# mke2fs -F -j mandriva.img
  • Finally, you could mount your image file on a mount point.
# mount -o loop mandriva.img $mymnt


You could now use this mount point like a physical partition's mount point and install Mandriva.

In the guest domain config file, disk option should replace phy:hdaX by file:path/of/file. Otherwise, other options are similar.

[edit]

Required additional domains' rpms

  • kernel-xenU: The one essential rpm for a guest domain is the XenoLinux kernel. If you installed your partition as a copy from the domain0 root partition this rpm is already installed. Else install it.
[edit]

Additional domain configuration

Before you can launch an additional domain, you should create a configuration file for guest domain. The following section describes this in detail. After that, the other sections aren't essential but contribute on the guest domain to a more proper boot and execution.

[edit]

Create a guest domain configuration file

  1. ConfigFile Two example can be used as a starting point in the /etc/xen directory. The following configuration file (call it mandriva) should be located in domain0's /etc/xen directory for a manual launch or in domain0's /etc/xen/auto directory for automatic launch by xendomains at system boot time.
kernel = "/boot/vmlinuz-2.6.12-18mdkxenU"
memory = 128
name = "Mandriva"
dhcp = "dhcp"
disk = [ 'phy:hda6,hda1,w', 'phy:hda7,hda5,w' ]
root = "/dev/hda1 ro"
extra = "3"
hostname = "mandriva2006"

This file shows the most common options used for a domain definition.

Here is a short description of each option:

  • kernel: Set this to the path of the kernel you compiled for use with Xen.
  • memory: Set this to the size of the guest domain's memory in megabytes.
  • name: Name of the additional domain.
  • dhcp: Uncomment the dhcp variable, so that the domain will receive its IP address from a DHCP server.
  • disk: List of block devices exported to the guest domain. In this example, physical partition hda6 takes the hda1 name in the additionnal domain and will be the root partition with the following root option. hda7 partition is a swap. If your disk is an image file you should replace phy:hdaX by file:path/of/file. The w= option explains the read-write right access on this partition. You could also use the =r option to access the partition read-only.
  • root: Specify the root device parameter on the kernel command line. We must take the partition scheme of the disk option.
  • extra: Extra string to append to the kernel command line (In this case, system starts in runlevel 3).
  • hostname: guest domain hostname.

Remark: there is one option by line. Use "#" for comment.

All modifications of the configuration in the next sections must be made in the guest domain and not in domain0 as before.

For this, we have two possibilities:

  1. Otherwise we launch the additional domain with the xm create Mandriva -c command and we modify guest domain directly in it.
  2. The last method consists of making a chroot command like this: chroot $mymnt. To exit the chroot shell make exit and don't forget to unmount the $mymnt mount point.

I retain the last method.

[edit]

/etc/inittab

If you do not want to have inopportunely messages as follows:

INIT: Id "2" respawning too fast: disabled for 5 minutes
INIT: Id "3" respawning too fast: disabled for 5 minutes
INIT: Id "4" respawning too fast: disabled for 5 minutes
INIT: Id "5" respawning too fast: disabled for 5 minutes
INIT: Id "6" respawning too fast: disabled for 5 minutes

then in your guest console, you should comment unused terminals in /etc/inittab file as follows:

1:2345:respawn:/sbin/mingetty tty1
#2:2345:respawn:/sbin/mingetty tty2
#3:2345:respawn:/sbin/mingetty tty3
#4:2345:respawn:/sbin/mingetty tty4
#5:2345:respawn:/sbin/mingetty tty5
#6:2345:respawn:/sbin/mingetty tty6
[edit]

xend and xendomains services

If you installed your system as a domain root filesystem copy as explained before, then xend and xendomains services are probably configured to start at boot time. It's totally useless for an additional domain. We disable these as follows:

# /etc/init.d/xend stop
# /etc/init.d/xendomains stop
# chkconfig xend off
# chkconfig xendomains off
[edit]

keytable service

After all, a little hint for avoid failure messages on keymap loading on guest domain boot. You should disable the keytable service with:

# chkconfig keytable off
# /etc/init.d/keytable stop

but it's not enough because the service is directly called in the /etc/rc.d/rc.sysinit file. We will comment out the three following lines (lines 869,870,871):

#if [ -x /etc/init.d/keytable -a -d /usr/lib/kbd/keymaps ]; then
#    /etc/init.d/keytable start
#fi

We take note that keytable< and numlock (following section) services are not necessary in additional domains because the domain0 has already initialised and all use the same keyboard.

[edit]

numlock service

Finally in the guest domain configuration, we have this message when the numlock service is started or when we log in:

KDGETLED: Argument invalide
Error reading current led setting. Maybe stdin is not a VT?

For stopping this problem, disable service as follows:

# /etc/init.d/numlock stop
# chkconfig numlock off

At this step, we have a Xen system which contains the domain0 and one or more clean additional domains, ready for execution. The following part describes usage of the Xen system.

[edit]

Manage guest domain

[edit]

Network configuration

This section explain the network bases and configuration in Xen.

[edit]

Networking

The network scheme is very simple. domain0 has a real Ethernet interface control. Each guest domain has a virtual network interface. In additionnal domain, this interface is a normal Ethernet interface but, the interface is named like vifX.Y with X as number interface (0 for eth0) and Y as number of it's own instance domain. All vif and eth interfaces are connected on xen-br0 bridge for network access.

For more details, this web site explains networking in detail: XenNetworking. Here virtual Ethernet interfaces, MAC addresses, bridging, routing, interface names and VLANs are explained in this documentation. It is rather complete.

Next section: configure the Xen interface and last section: correct a little problem with Mandriva's bridge.

[edit]

Configuring ifcfg-xen

Xen RPM installs its kernel and configuration files but you should configure yourself the network configuration of the Xen interface. The simple way to create a Xen network configuration is copying the xen-br0 config file and change the device entry in the file or create /etc/sysconfig/network-scripts/ifcfg-xen with following content:

DEVICE=xen
BOOTPROTO=dhcp
ONBOOT=yes

You should verify in the Ethernet interface configuration (typically in the file ifcfg-eth0) that the MII_NOT_SUPPORTED option is present and set to yes. This permits starting the network without checking for an Ethernet link status (that is in the guest case).

[edit]

Bridge conflict

By default, Mandriva Linux 2006 sets up a bridge named br0. But Xen sets up a bridge too, named xen-br0. You can see the bridge on your domain0 with this command:

# brctl show
bridge name     bridge id          STP enabled      interfaces
xen-br0         8000.fefffffffffff no               peth0
                                                    vif0.0

There, two bridges could create a conflict and you could have network problems. To avoid this:

# chkconfig bridge off
# /etc/init.d/bridge stop
[edit]

The xm tool

The xm tool provides many options for managing domains. I present the more useful commands and bases of this tool:

[edit]

Start guest domain

Before you start a domain, you should create a config file for it. See here.

xm simplified syntax is: xm command [options].

For starting an additional domain:

# xm create -c mandriva

The create command starts a new domain instance. The -c option specifies that the console domain should output immediately and mandriva is a name of the additional domain you would start. You can now see a list of running domains:

# xm list

You can see detailed syntax of commands and options with the xm help command.

[edit]

Using Xen console

If you don't specify the -c option at start time you could use console too.

# xm console mandriva

Mandriva is the name of the guest domain target. You could replace it by guest id displayed in xm list.

To quit from a guest console, just type ctrl+].

[edit]

Guest domains save and restore

The administrator of a Xen system could suspend a virtual machine in current state into a disk file in domain0, allowing it to be resumed at a later time.

For example you can suspend a domain called mandriva to disk using the command:

# xm save mandriva mandriva.chk

This will stop the domain named mandriva and save its current state into a file called mandriva.chk.

To resume execution of this domain, use the xm restore command:

# xm restore mandriva.chk

This will restore the state of the domain and resume its execution. The domain will restart as before and the console may be reconnected using the xm console command, as described earlier.

[edit]

VNC

Remote control of a VNC virtalised system.

  • configure network between domain0 and guest domain (if not already done)
  • start X session on domain0
# startx
  • on guest domain, log in and start VNC server
# vncserver
  • on domain0, save guest domain registered password
# vncpasswd
  • on domain0, take VNC guest domain control
# vncviewer -geometry 800x600 -passwd /root/.vnc/passwd 192.168.1.1:1
[edit]

Possible errors and solutions

  • The nics option is deprecated: replace "nics option" by vif = ["].
  • features is an invalid keyword argument for this function
# cd ~/xen-2.0
# make uninstall
# cd ~/xen-3.0
# make install
# reboot
  • Error: Error creating domain (12, 'Cannot allocate memory'): add dom0_mem option on Xen command line in the grub.conf entry.
  • Error: Error creating domain: Kernel image does not exist: /boot/vmlinuz-2.6.12-18mdkxenU: install kernel-xenU-2.6.12 package.
[edit]

Links

Retrieved from "http://wiki.mandriva.com/Development/Howto/Xen"
Personal tools