Version 14 (modified by ibaldin, 9 years ago)

--

Setting up Eucalyptus 1.5.2 with ORCA 2.1

This page explains how to setup and test Euca 1.5.2 for ORCA. We will begin by setting up a stock distribution, testing it, then applying ORCA-specific patches.

The idea is the Eucalyptus will set up slivers consisting of one or more VMs hanging off vlans, whose IDs orca specified at creation time. Then these slivers with vlans can be dynamically stitched to other slivers into a complete slice. Using Eucalyptus simplifies substrate setup compared to Xen.

Setting up stock Eucalyptus 1.5.2

Hardware setup

You will need several hosts with dual interfaces - one interface on 'management' network and one on 'data plane' that will be stitched into ORCA slices. At RENCI this setup is implemented by having each host its eth1 on BEN management network (192.168.xx.xx address space) and eth0 connected into BEN with unassigned IP addresses.

Each host should support hardware virtualization and be able to run kvm. The cluster will consist of a single head node and multiple compute nodes. The head node requires substantial disk space to store all VM filesystem images (if you plan to support many options). The head node can also act as a compute node and instantiate VMs on itself. These instructions however presume that the head node is not performing this function.

The dataplane interfaces of the cluster should be plugged in into an ORCA-controllable switch (a Cisco 6509 in our case) to allow for the mapping of Euca-created vlans to other vlan segments.

Software pre-requisutes

  1. Ubuntu jaunty basic server install
  2. kvm and libvirt (including libvirt-bin). Ubuntu favors kvm over Xen. KVM requires hardware virtualization support in your CPU!
  3. ntp (Euca instructions suggest using open-ntp, however there is no reason not to use the Ubuntu 9.04 stock ntpd3 server
  4. vconfig tools (to enable creating tagged interface)
  5. brctl tools (to enable creating bridges)

Testing software pre-requisites

The notes here are either for the head node [HN], the compute nodes [CN], or for all [ALL]

  1. [ALL] Install and test ntp. Run ntpdc and verify the output is sane (substitute your own NTP server):
    $ apt-get install ntp
    $ echo server clock3.unc.edu >> /etc/ntp.conf
    $ /etc/init.d/ntp restart
    $ ntpdc
    

line to /etc/ntp.conf and restarting ntpd

  1. [ALL] Test vconfig and brctl:
    $ vconfig add eth0 10
    $ ifconfig eth0.10
    $ vconfig rem eth0.10
    $ brctl show 
    
  2. [CN] Make sure kvm is OK. If you receive a message about a problem with a kernel module either your CPU does not support hardware virtualization, or it is disabled in the BIOS. In the latter case, edit the BIOS setting and try again:
    $ /etc/init.d/kvm restart
    
  3. [CN] Make sure libvirtd is running:
    $ /etc/init.d/libvirt-bin restart
    $ virsh list
    
  4. [CN] identify or create a default bridge for kvm/xen to use. Xen by default creates a bridge (xenbr0). KVM requires that a bridge is manually setup. On Ubuntu this means adding
    auto br0
    iface br0 inet manual
    	bridge_ports eth0
    	bridge_stp off
    	bridge_maxwait 0
    

to /etc/network/interfaces. NOTE: in this setup eth0 is the dataplane interface facing into BEN. It remains unconfigured. The management interface is eth1 and is not shown here - it has a static configuration. Restart networking and verify that bridge br0 exists and eth0 is part of it, verify that br0 and eth0 are in the UP:

$ brctl show
$ ifconfig br0
$ ifconfig eth0

Refer to the discussion here about the significance of having the correct bridge setup.

  1. [HN] Install DHCP server. It does not have to be configured or running. Euca will start it when needed.
    $ apt-get install dhcp3-server
    

Installing Eucalyptus

Follow the instructions here. Be sure to select the right packages for your architecture. BEN cluster uses amd64 packages for eucalyptus and euca2ools.

Configuring Eucalyptus

We will configure Euca to run in MANAGED network mode to enable dynamic VLAN creation. This section only identifies entries in the /etc/eucalyptus.conf that differ from the default or need to be verified: [HN]

VNET_INTERFACE="eth0"
VNET_BRIDGE="br0"
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
VNET_DHCPUSER="dhcpd"
VNET_MODE="MANAGED"
VNET_SUBNET="172.16.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="192.168.201.254"
VNET_ADDRSPERNET="32"
#VNET_PUBLICIPS="your-free-public-ip-1 your-free-public-ip-2 ..."
#VNET_MODE="SYSTEM"

[CN]

VNET_INTERFACE="eth0"
VNET_BRIDGE="br0"
VNET_MODE="MANAGED"
#VNET_MODE="SYSTEM"

Undoing an install

When things don't seem to work, fear not, there is a way to start from scratch:

  1. Stop the euca daemons:

[HN]

$ /etc/init.d/eucalyptus-cc stop
$ /etc/init.d/eucalyptus-cloud stop

[CN]

$ /etc/init.d/eucalyptus-nc stop
  1. Remove eucalyptus packages (including config directories, if possible)

[HN]

$ dpkg --purge eucalyptus-cloud
$ dpkg --purge eucalyptus-cc
$ dpkg --purge eucalyptus-gl
$ dpkg --purge eucalyptus-common
$ dpkg --purge eucalyptus-javadeps

[CN]

$ dpkg --purge eucalyptus-cc
$ dpkg --purge eucalyptus-gl
$ dpkg --purge eucalyptus-common
  1. Remove user eucalyptus from the system
    $ userdel -r eucalyptus
    $ groupdel eucalyptus
    
  2. Remove remnants of config and log directories
    $ rm -rf /etc/eucalyptus
    $ rm -rf /var/log/eucalyptus
    
  3. Sometimes you may need to fix dpkg state
    $ vi /var/lib/eucalyptus/statoverride
    

and remove the line that mentions 'eucalyptus'

  1. Start over

References

KVM Networking

Euca install on Jaunty

euca-group-add bug

Attachments