Version 15 (modified by ibaldin, 8 years ago)

--

Creating a customized xCAT image for a Eucalyptus worker node

Overview

This document describes how to create a custom xCAT image that will contain a Eucalyptus worker node and how to connect it to the already existing Eucalyptus master node.

Required modifications for Centos 5.5

Assuming you are starting from a Centos 5.5 here are the modifications that are required to turn a stock Centos 5.5 install into a Eucalyptus worker. Presumably a tar file containing Eucalyptus worker already exists. By default Eucalyptus is presumed to be installed under /opt/eucalyptus-2.0

  • Install prerequisites (admittedly this list contains both binary and development prerequisites):
    # yum install -y libvirt-devel curl-devel httpd httpd-devel apr-devel openssl-devel dhcp libxml2 libxml2-devel gnutls gnutls-devel xen-devel libgcrypt-devel zlib-devel perl-Convert-ASN1 perl-Crypt-OpenSSL-RSA perl-Crypt-OpenSSL-Random chkfontpath scsi-target-utils fuse-libs swig gcc
    
  • Install KVM (if hardware supports it). Xen otherwise. This writeup is geared to KVM.
    # yum install kvm
    
  • Make sure libvirtd is runnig and 'virsh list' can execute:
    # virsh list
     Id Name                 State
    ----------------------------------
    #
    
  • Run 'virsh capabilities' to see that 'kvm' virtualization is supported.
  • Symlink /usr/libexec/qemu-kvm to /usr/bin/kvm. Eucalyptus looks for the latter, while default Centos comes with former.
  • Make sure bridge named 'br0' is created at boot time. This is separate from virbr0 created by libvirt by default. br0 needs to carry the management IP address of the node and eth0 should be a member of this bridge:
    # ifconfig br0
    br0       Link encap:Ethernet  HWaddr 00:1D:09:29:FF:DA  
              inet addr:192.168.201.43  Bcast:192.168.201.255  Mask:255.255.255.0
              inet6 addr: fe80::21d:9ff:fe29:ffda/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:7621193 errors:0 dropped:0 overruns:0 frame:0
              TX packets:1793760 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0 
              RX bytes:9385876051 (8.7 GiB)  TX bytes:680733217 (649.1 MiB)
    # [root@euca-c1 libvirt]# brctl show
    bridge name	bridge id		STP enabled	interfaces
    br0		8000.001d0929ffda	no		eth0
    virbr0		8000.000000000000	yes
    

To do this create /etc/sysconfig/network-scripts/ifcfg-br0 (address, netmask and gateway should be same as was eth0):

# Euca default bridge
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.201.43
NETMASK=255.255.255.0
GATEWAY=192.168.201.1

Also, modify /etc/sysconfig/network-scripts/ifcfg-eth0 to be (Don't copy the MAC address! ):

# Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet
DEVICE=eth0
HWADDR=00:1D:09:29:FF:DA
TYPE=Ethernet
BRIDGE=br0
ONBOOT=yes
  • User and group 'eucalyptus' needs to be created. Group 'libvirt' needs to be created and user eucalyptus must be made a member. Also, libvirt configuration (/etc/libvirt/libvirtd.conf) needs to be modified:
    #################################################################
    #
    # UNIX socket access controls
    #
    
    # Set the UNIX domain socket group ownership. This can be used to
    # allow a 'trusted' set of users access to management capabilities
    # without becoming root.
    #
    # This is restricted to 'root' by default.
    unix_sock_group = "libvirt"
    
    # Set the UNIX socket permissions for the R/O socket. This is used
    # for monitoring VM status only
    #
    # Default allows any user. If setting group ownership may want to
    # restrict this to:
    #unix_sock_ro_perms = "0777"
    
    # Set the UNIX socket permissions for the R/W socket. This is used
    # for full management of VMs
    #
    # Default allows only root. If PolicyKit is enabled on the socket,
    # the default will change to allow everyone (eg, 0777)
    #
    # If not using PolicyKit and setting group ownership for access
    # control then you may want to relax this to:
    unix_sock_rw_perms = "0770"
    
    # Set the name of the directory in which sockets will be found/created.
    #unix_sock_dir = "/var/run/libvirt"
    
  • Copy Eucalyptus tar file onto the node and untar it so eucalyptus is in /opt/eucalyptus-2.0
  • Run ldconfig (the tar file should also contain /etc/ld.so.conf.d/iniparser) so iniparser libraries are available
  • Run setup for eucalyptus :
    /opt/eucalyptus-2.0/usr/sbin/euca_conf -d /opt/eucalyptus-2.0 --hypervisor kvm --instances /opt/eucalyptus-2.0-instances --user eucalyptus --setup
    
  • The last command overwrites the eucalyptus.conf config file, so it needs to be put back into place (e.g. from a previously saved tmp, or should be present in tar file under alternative name
  • For Centos 5.5 KVM out of the box does not support SCSI bus in guests. The easiest way around is to tell KVM to use IDE bus instead by modifying /opt/eucalyptus-2.0/usr/share/eucalyptus/gen_kvm_libvirt_xml:
    if ( $use_virtio_root ) {
        print <<EOF;
            <disk type='file' device='disk'>
                <source file='BASEPATH/disk'/>
                <target dev='vda' bus='virtio'/>
            </disk>
    EOF
    } else {
        print <<EOF;
            <disk type='file'>
                <source file='BASEPATH/disk'/>
                <target dev='sda' bus='ide'/>  <--- ADD bus='ide'
            </disk>
    EOF
    }
    
  • Start node controller
  • Make it known to the master
    [euca-master]# /opt/eucalyptus-2.0/usr/sbin/euca-conf --register-nodes  "node name or IP"
    
  • Observe the result using euca-describe-availability-zones verbose