Version 49 (modified by shuang, 9 years ago)

--

Setting up Eucalyptus 1.6.2 with ORCA 2.x

This page explains how to setup and test Euca 1.6.2 for ORCA. Most of the steps are similar with https://geni-orca.renci.org/trac/wiki/Eucalyptus-1.5.2-Setup. However, since the servers we have do not support hardware virtualization, we assume XEN is used.

Debian

XEN installation on Debian Lenny

According to http://open.eucalyptus.com/wiki/EucalyptusInstall_v1.6, Euca 1.6.2 package is only available on Debian Squeeze. Unfortunately, Debian Squeeze does not support XEN well. Therefore we chose to use Lenny and install Euca from source.

XEN installation on Lenny is fairly simple:

  1. # apt-get install xen-utils
  2. # apt-get install xen-tools
  3. # apt-get installxen-linux-system-2.6.26-2-xen-amd64 1.# reboot into xen

Installing Eucalyptus 1.6.2 from source

  1. use dpkg --get-selections to make sure libc and pthreads development files are installed
  2. # apt-get install gcc make apache2-threaded-dev ant openjdk-6-jdk libvirt-dev libcurl4-gnutls-dev dhcp3-server vblade apache2 unzip curl vlan bridge-utils libvirt-bin sudo vtun (some of them may not be necessary for compute node installation, e.g., dhcp3-server, vtun, etc)
  3. get tarballs:
    wget http://eucalyptussoftware.com/downloads/releases/eucalyptus-1.6.2-src.tar.gz
    wget http://eucalyptussoftware.com/downloads/releases/eucalyptus-1.6.2-src-deps.tar.gz
    
  4. make sure JAVA_HOME is set, for Debian, if you use the default, it's something like: export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
  5. follow the steps as in http://open.eucalyptus.com/wiki/installing-eucalyptus-source-16, make sure the following dependencies are met:
    1. Axis2
    2. Axis2/C
    3. Rampart/C
  6. # useradd eucalyptus
  7. run virsh list as eucaplytus, if it does not work -
    1. # adduser eucalyptus libvirt
    2. edit /etc/xen/xend-config.sxp, make sure '(xend-unix-server yes)' and '/var/lib/xend/xend-socket' set, use lsof to verify xend-socket is open

Configuring Network

We assume Xen3.2 is installed. For more detailed information, please refer to http://open.eucalyptus.com/wiki/EucalyptusNetworking_v1.6. The notes here are either for the head node (aka front-end) [HN], the compute nodes [CN], or for all [ALL]

  1. [ALL] Install and test ntp. Run ntpdc and verify the output is sane (substitute your own NTP server):
    $ sudo apt-get install ntp
    $ sudo echo server clock3.unc.edu >> /etc/ntp.conf
    $ sudo /etc/init.d/ntp restart
    $ ntpdc
    
  2. Make sure the dataplane interface (although unconfigured) is UP
    # ifconfig eth0 up
    
  3. [ALL] Test vconfig and brctl:
    # vconfig add eth0 10
    # ifconfig eth0.10
    # vconfig rem eth0.10
    # brctl show 
    
  4. [CN] Make sure libvirtd is running:
    # /etc/init.d/libvirt-bin restart
    # virsh list
    
  5. [CN] identify or create a default bridge for xen to use. Some versions of Xen by default create a bridge (called xenbr0 or eth0 on xen3.2). If your version does not, you can do this manually. One option is:
    edit /etc/xen/xend-config.sxp and make sure (network-script network-bridge) is set, then restart xend.
    
    Or, add
    auto br0
    iface br0 inet manual
    	bridge_ports eth0
    	bridge_stp off
    	bridge_maxwait 0
    

to /etc/network/interfaces. NOTE: in this setup eth0 is the dataplane interface facing into BEN. It remains unconfigured. The management interface is eth1 and is not shown here - it has a static configuration. Restart networking and verify that bridge br0 exists and eth0 is part of it, verify that br0 and eth0 are in the UP:

# brctl show
# ifconfig br0 up
# ifconfig eth0 up

We will configure Euca to run in MANAGED network mode to enable dynamic VLAN creation. This section only identifies entries in the /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf (assume eucalyptus is installed in /opt/eucalyptus) that differ from the default or need to be verified:

[HN]

HYPERVISOR="xen"
VNET_PUBINTERFACE="eth1"
VNET_PRIVINTERFACE="eth0"
VNET_BRIDGE="br0"
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
VNET_MODE="MANAGED"
VNET_SUBNET="172.16.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="192.168.201.254"
VNET_ADDRSPERNET="32"
#VNET_PUBLICIPS="your-free-public-ip-1 your-free-public-ip-2 ..."
#VNET_MODE="SYSTEM"

[CN]

HYPERVISOR="xen"
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"
VNET_BRIDGE="br0"
VNET_MODE="MANAGED"
#VNET_MODE="SYSTEM"

Note 1: VNET_SUBNET, VNET_NETMASK and VNET_ADDRSPERNET have to be set up properly, refer to the discussion How Eucalyptus allocates VLAN tags in https://geni-orca.renci.org/trac/wiki/Eucalyptus-1.5.2-Setup.

Note 2: the eucalptus.conf says that CN's VNET_PUBINTERFACE and VNET_PRIVINTERFACE can be the name of either the bridge (br0 in our case) or the physical interface, however, using the bridge name somehow does not work for me.

Configuring Eucalyptus

  1. Euca components. Assume Euca is installed /opt/eucalyptus,

[ALL]

# /opt/eucalyptus/usr/sbin/euca_conf -d $EUCALYPTUS --hypervisor xen --instances /usr/local/eucalyptus --user eucalyptus --setup
  1. Euca startup scripts

[HN]

# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud
# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc

[CN]

# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

Running Eucalyptus

  1. [HN]
    # $EUCALYPTUS/usr/sbin/euca_conf --enable cloud --enable walrus --enable sc
    # /etc/init.d/eucalyptus-cloud restart
    # /etc/init.d/eucalyptus-cc restart
    

Note: eucalyptus-cc supports cleanstart/cleanstop/cleanrestart, when you change the conf file, you may need to do a clean restart.

  1. [CN]
    # /etc/init.d/eucalyptus-nc restart
    
    1

Eucalyptus first time setup

  1. [HN] registering Eucalyptus components
    $EUCALYPTUS/usr/sbin/euca_conf --register-walrus <front end IP address>
    $EUCALYPTUS/usr/sbin/euca_conf --register-cluster <clustername> <front end IP address>
    $EUCALYPTUS/usr/sbin/euca_conf --register-sc <clustername> <front end IP address>
    $EUCALYPTUS/usr/sbin/euca_conf --register-nodes "<Node 0 IP address> <Node 1 IP address> ... <Node N IP address>"
    
  2. point web browser to https://head-node-ip:8443 (refer to http://open.eucalyptus.com/wiki/first-time-setup-16 for details)
  3. Eucalyptus attempts to guess your HN external ip, confirm it's correct.
  4. [HN] From the 'Credentials' tab, download zipped credentials and do
    mkdir $HOME/.euca
    unzip euca2-admin-x509.zip -d $HOME/.euca
    . $HOME/.euca/eucarc
    

Installing Euca2tools [HN]

  1. get 1.2 source for Euca 1.6.2 and dependencies
    wget http://eucalyptussoftware.com/downloads/releases/euca2ools-1.2.tar.gz
    wget http://eucalyptussoftware.com/downloads/releases/euca2ools-1.2-src-deps.tar.gz
    
  2. install python stuff
    # apt-get install python-dev swig help2man libssl-dev
    
  3. install boto
    tar zvxf euca2ools-$VERSION-src-deps.tar.gz
    cd euca2ools-$VERSION-src-deps
    tar zxvf boto-1.8d.tar.gz
    cd boto-1.8d
    sudo python setup.py install
    cd ..
    
  4. install M2Crypto
    tar zxvf M2Crypto-0.19.1.tar.gz
    cd M2Crypto-0.19.1
    sudo python setup.py install
    cd ..
    
  5. install Euca2ools in /usr/local/bin, adding it to your $PATH, if necessary
    tar zxvf euca2ools-$VERSION.tar.gz
    cd euca2ools-$VERSION
    sudo make
    export PATH=/usr/local/bin:$PATH  # not necessary on most installations
    

Eucalyptus Management

Follow the steps in http://open.eucalyptus.com/wiki/EucalyptusImageManagement_v1.6, briefly:

  1. Adding images
  2. Associating kernels and ramdisks with instances
  3. Example:
    wget http://eucalyptussoftware.com/downloads/eucalyptus-images/euca-debian-5.0-x86_64.tar.gz
    tar zxvf euca-debian-5.0-x86_64.tar.gz
    euca-bundle-image -i euca-debian-5.0-x86_64/xen-kernel/vmlinuz-2.6.27.21-0.1-xen --kernel true
    euca-upload-bundle -b debian-kernel-bucket -m /tmp/vmlinuz-2.6.27.21-0.1-xen.manifest.xml 
    euca-register debian-kernel-bucket/vmlinuz-2.6.27.21-0.1-xen.manifest.xml
    (set the printed eki to $EKI)
    euca-bundle-image -i euca-debian-5.0-x86_64/xen-kernel/initrd-2.6.27.21-0.1-xen --ramdisk true
    euca-upload-bundle -b debian-ramdisk-bucket -m /tmp/initrd-2.6.27.21-0.1-xen.manifest.xml
    euca-register debian-ramdisk-bucket/initrd-2.6.27.21-0.1-xen.manifest.xml
    (set the printed eri to $ERI)
    euca-bundle-image -i euca-debian-5.0-x86_64/debian.5-0.x86-64.img --kernel $EKI --ramdisk $ERI
    euca-upload-bundle -b debian-image-bucket -m /tmp/debian.5-0.x86-64.img.manifest.xml
    euca-register debian-image-bucket/debian.5-0.x86-64.img.manifest.xml 
    

CentOS5.4

Installing Eucalyputs 1.6.2

  1. export VERSION=1.6.2
  2. yum install -y ntp
  3. yum install -y java-1.6.0-openjdk ant ant-nodeps dhcp bridge-utils httpd
  4. yum install -y xen
    sed --in-place 's/#(xend-http-server no)/(xend-http-server yes)/' /etc/xen/xend-config.sxp 
    sed --in-place 's/#(xend-address localhost)/(xend-address localhost)/' /etc/xen/xend-config.sxp
    /etc/init.d/xend restart
    

Testing Eucalyptus

  1. [HN] Create a key pair that will be injected into the VM to be created so user can ssh to it:
    euca-add-keypair mykey >mykey.private
    ('mykey' is the name for the key in Eucalyptus, 'mykey.private' is the file to be used by ssh)
    
    chmod 0600 mykey.private
    
    euca-run-instances --addressing private -k mykey -n <number of instances to start> <emi-id> 
    
    euca-describe-instances
    (should now show the instance)
    
    ssh -i mykey.private root@<accessible-instance-ip>
    (this should get you directly into the VM, i.e., no password asked!)
    
  2. [HN as root] If client machines aren't showing up, try restarting cloud controller and portal
    # /etc/init.d/eucalyptus-cc restart
    # /etc/init.d/eucalyptus-cloud restart
    

Troubleshooting

  1. Check if the bridges are created (eucabrXX) in the compute nodes:
    # brctl show
    

If something wrong, this procedure can be done manually so you can check the correctness of every step:

# vconfig add eth0 10
# ifconfig eth0.10 up
# brctl addbr testbr10 
# brctl addif testbr10 eth0.10
# ifconfig testbr10 10.0.0.2 up

Do the same on front-end and make testbr10 10.0.0.1, make sure it can ping 10.0.0.2. Then undo what we just did for both front-end and the node:

# ifconfig testbr10 down
# ifconfig eth0.10 down
# brctl delbr testbr10
# vonfig rem eth0.10
  1. run
    euca-describe-availability-zones verbose
    

to make sure there is enough resources for creating the VM. If not you could add more compute nodes to the cluster.

  1. Check the VMs are created in machines (may have to hunt for them since you don't know which specific compute node a VM will be created on):
    # virsh list
    

You can use

$ virsh console domain-id

to monitor a VM is successfully started.

  1. Check the logs
  • [HN] /var/log/eucalyptus/cc.log
  • [CN] /var/log/eucalyptus/nc.log