Version 9 (modified by jonmills, 8 years ago)

--

Installing Euca 2.0.3 from RPM on CentOS

Overview

Goal of this document is to show site administrators a way to quickly install a Eucalyptus cluster in a fashion that leverages well-known tools such as DHCP, PXE, Kickstart, and Yum, to create a configuration that is robust and highly repeatable -- allowing one to easily scale the cluster by adding new worker nodes with a minimum of hassle.

It should work like this:

  1. Power on machine, and boot into PXE
  2. Wait for DHCP response, which should give it a 'next-server' pointing to the location of your tftpserver, from which it will Kickstart
  3. Kickstart file should be pre-configured with:
    1. Yum repository data, linking to repos containing pertinent packages
    2. Kickstart %packages section contains names of Euca-related packages to install
    3. Kickstart %post section performs some useful initial configuration

Assumptions

  • You know how to use yum repositories
  • You know how to PXE boot and install a linux RPM-based distro via Kickstart
  • You are installing onto CentOS 5.4, 5.5, or 5.6

Duke NEuca Yum Repository

The Euca 2.0.3 (with NEuca patches) RPM files were compiled by Victor Orlikowski at Duke. He has graciously provided a public Yum repository for accessing them. You may want to mirror that repo to your own local site, however.

duke-neuca.repo:

[duke-neuca]
name=CentOS NEuca Repo (at Duke CS) $releasever - $basearch
baseurl=http://dbc1-16.nicl.cs.duke.edu/repos/centos/5/$basearch/
enabled=1
gpgcheck=0

With the above file installed in your /etc/yum.repos.d/, a Yum search for 'euca' should produce something like this:

[root@euca ~]# yum search --disablerepo=* --enablerepo=duke-neuca euca
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
========================================================= Matched: euca =========================================================
euca-axis2.noarch : Elastic Utility Computing Architecture - axis2
euca-axis2c.x86_64 : Elastic Utility Computing Architecture - axis2c
euca-rampartc.x86_64 : Elastic Utility Computing Architecture - rampartc
euca2ools.noarch : Elastic Utility Computing Architecture Command-Line Tools
eucalyptus.x86_64 : Elastic Utility Computing Architecture
eucalyptus-cc.x86_64 : Elastic Utility Computing Architecture - cluster controller
eucalyptus-cloud.x86_64 : Elastic Utility Computing Architecture - cloud controller
eucalyptus-common-java.x86_64 : Elastic Utility Computing Architecture - ws java stack
eucalyptus-gl.x86_64 : Elastic Utility Computing Architecture - log service
eucalyptus-nc.x86_64 : Elastic Utility Computing Architecture - node controller
eucalyptus-sc.x86_64 : Elastic Utility Computing Architecture - storage controller
eucalyptus-walrus.x86_64 : Elastic Utility Computing Architecture - walrus
neuca-iniparser.x86_64 : Elastic Utility Computing Architecture - iniparser
  • At the time of this writing, the RPMS seen here install Eucalyptus 2.0.3.
  • Note the presence of the neuca-iniparser...

EPEL Yum Repository

EPEL (Extra Packages for Enterprise Linux) is a yum repo maintained by the Fedora Project. It will provide some useful packages for us, including the euca2ools package that is necessary for a completely functional Euca cluster. http://download.fedora.redhat.com/pub/epel/5/

[root@euca ~]# yum search --disablerepo=* --enablerepo=epel euca2ools
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * epel: archive.linux.duke.edu
================================================================ Matched: euca2ools ================================================================
euca2ools.noarch : Elastic Utility Computing Architecture Command-Line Tools

PXE Boot Configuration

You need to have a functioning PXE environment. Describing a thorough setup of that is beyond the scope of this document. In our example, we ran the default tftp server in CentOS 5, with the tftp root in /tftpboot. (Paths in the file are relative to that location.) Our PXE config lives in /tftpboot/pxelinux.cfg/default and looks like:

label CENTOS56KSVMx64
        menu label CentOS 5.6 (X86_64 Kick Start Euca Worker Node)
        kernel CentOS/CentOS56/x86_64/vmlinuz
        append initrd=CentOS/CentOS56/x86_64/initrd.img noipv6 ramdisk_size=150000 lang=en devfs=nomount pxe \ 
        kssendmac selinux=0 keymap=us ip=dhcp method=http://172.25.8.20/CentOS56-x86_64/disk1 \
        ks=http://172.25.8.20/cfg/centos/CentOS56-x86_64/ks-euca.cfg ksdevice=eth0

Kickstart Configuration

In your kickstart cfg file, add instructions to use additional repositories. Put this section in the main body of the file, above the %packages section:

##
# Additional YUM repositories
##

repo --name=duke-neuca --baseurl=http://dbc1-16.nicl.cs.duke.edu/repos/centos/5/x86_64/
repo --name=epel --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-5&arch=x86_64

In addition these, you may want to build your own local repository to hold other RPMS that are required dependencies for Euca. A good example is the Sun Java JDK package (typically just 'jdk' if specified in a package list).

Package list

This is a "nice", but certainly not minimal, set of packages one might install for a Euca worker node. It has dependencies for Euca, as well as the 'jdk' package installed from a local (private) repo.

%packages --resolvedeps
@admin-tools
@editors
@text-internet
@x-software-development
@gnome-desktop
@core
@base
@java
@java-development
@base-x
@graphics
@printing
@development-libs
@development-tools
@graphical-internet
libgnome-java
libgtk-java
libgconf-java
mesa-libGLU-devel
kexec-tools
xorg-x11-server-Xnest
xorg-x11-server-Xvfb
agg
libsane-hpaio
imake
net-snmp
net-snmp-devel
sysstat
compat-libstdc++-33
compat-libstdc++-33
xinetd
nss-mdns
perl-LDAP
perl-IO-Socket-SSL

perl-Net-SSLeay
perl-XML-Parser
libXp
openmotif
openmotif22
emacs
emacs-nox
xfig
lynx
tftp
cpufreq-utils
iscsi-initiator-utils
x86info
am-utils
hwbrowser
iptraf
lsscsi
mrtg
mt-st
net-snmp-utils
nmap
nmap-frontend
screen
yum-utils
yumex
-tog-pegasus
-sysreport
-evolution-data-server
-evolution-webcal
-evolution-connector
-evolution
-ekiga

# Things wanted specifically by eucalyptus
# !! jdk-1.6.0_26-fcs must be provided by YOU in a local repo !!
jdk-1.6.0_26-fcs
java-1.6.0-openjdk-devel
ant
ant-nodeps
libvirt
libvirt-devel
curl-devel
httpd
httpd-devel
apr-devel
openssl-devel
dhcp
libxml2
libxml2-devel
gnutls
gnutls-devel
xen-devel
libgcrypt-devel
zlib-devel
perl-Convert-ASN1
perl-Crypt-OpenSSL-RSA
perl-Crypt-OpenSSL-Random
perl-Crypt-X509
chkfontpath
scsi-target-utils
fuse-libs
swig
vblade

# Virtualization things
# (Installs KVM but not Xen)
etherboot-roms-kvm
etherboot-zroms-kvm
kmod-kvm
kmod-kvm-debug
kvm
kvm-qemu-img
kvm-tools
python-virtinst
sblim-gather-plugins-virt
sblim-gather-provider-plugins-virt
virt-manager

# Install custom
epel-release

# Install Euca worker node components
eucalyptus-nc
euca2ools

%Post section

In our Kickstart %post macro, we'll do some initial configuration:

# Libvirtd will, by default install a 'virbr0' but it's not a good idea to use that with Euca
# Instead, we'll create a 'br0' that uses 'eth0'.
# This expects you are controlling network configuration with DHCP
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 <<EOF
DEVICE=eth0
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0
EOF

cat > /etc/sysconfig/network-scripts/ifcfg-br0 <<EOF
# Euca default bridge
DEVICE=br0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=dhcp
DHCPCLASS=
EOF

# Configure java
# This depends upon you having installed the sun jdk in the %packages section from your own Yum repo...
/usr/sbin/alternatives --install /usr/bin/java java /usr/java/jdk1.6.0_26/jre/bin/java 1
/usr/sbin/alternatives --set java /usr/java/jdk1.6.0_26/jre/bin/java

# Disable euca at boot....at least until you've tested it
/sbin/chkconfig eucalyptus-nc off

# Default firewall allows SSH, Avahi, NTP, DHCP, and all traffic on local subnet (for Euca).
# Note that without '-A FORWARD -m physdev  --physdev-is-bridged -j ACCEPT' Euca nodes cannot DHCP.
cat > /etc/sysconfig/iptables <<EOF
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [249:77576]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT 
-A FORWARD -m physdev  --physdev-is-bridged -j ACCEPT 
-A FORWARD -j RH-Firewall-1-INPUT 
-A RH-Firewall-1-INPUT -i lo -j ACCEPT 
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT 
-A RH-Firewall-1-INPUT -p esp -j ACCEPT 
-A RH-Firewall-1-INPUT -p ah -j ACCEPT 
-A RH-Firewall-1-INPUT -d 224.0.0.251 -p udp -m udp --dport 5353 -j ACCEPT 
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT 
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT 
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT 
-A RH-Firewall-1-INPUT -s 152.54.10.0/255.255.255.192 -p udp -m state --state NEW -m udp --dport 68 -j ACCEPT 
-A RH-Firewall-1-INPUT -p udp -m state --state NEW -m udp --dport 123 -j ACCEPT 
-A RH-Firewall-1-INPUT -s 152.54.10.0/255.255.255.192 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited 
COMMIT
EOF

# Configure hypervisor for Euca
# You'll need a correctly configured libvirtd.conf on a webserver somewhere to grab with wget
/bin/mkdir -p /home/eucalyptus
/bin/chown 500:500 /home/eucalyptus/
/bin/chmod 755 /home/eucalyptus/
/bin/echo "libvirt:x:499:eucalyptus" >> /etc/group
mv /etc/libvirt/libvirtd.conf /etc/libvirt/libvirtd.conf.orig
/usr/bin/wget -c --directory-prefix=/etc/libvirt http://$INSTALL_SITE/extras/libvirtd.conf

# You'll get errors if you don't create this link
/bin/ln -s /usr/libexec/qemu-kvm /usr/bin/kvm

# Configure loop devices
/bin/echo 'options loop max_loop=255' >> /etc/modprobe.conf

Attachments