Changes between Version 48 and Version 49 of Eucalyptus-1.6.2-Setup

Show
Ignore:
Timestamp:
06/23/10 12:16:35 (9 years ago)
Author:
shuang (IP: 152.54.6.10)
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Eucalyptus-1.6.2-Setup

    v48 v49  
    99XEN installation on Lenny is fairly simple: 
    1010 
    11  1. sudo apt-get install xen-utils 
    12  1. sudo apt-get install xen-tools 
    13  1. sudo apt-get installxen-linux-system-2.6.26-2-xen-amd64 
    14  1. sudo reboot into xen 
     11 1. # apt-get install xen-utils 
     12 1. # apt-get install xen-tools 
     13 1. # apt-get installxen-linux-system-2.6.26-2-xen-amd64 
     14 1.# reboot into xen 
    1515 
    1616 
    1717=== Installing Eucalyptus 1.6.2 from source === 
    1818 1. use dpkg --get-selections to make sure libc and pthreads development files are installed 
    19  1. sodu apt-get install gcc make apache2-threaded-dev ant openjdk-6-jdk   libvirt-dev libcurl4-gnutls-dev dhcp3-server vblade apache2 unzip curl vlan  bridge-utils libvirt-bin sudo vtun (some of them may not be necessary for compute node installation, e.g., dhcp3-server, vtun, etc) 
     19 1. # apt-get install gcc make apache2-threaded-dev ant openjdk-6-jdk   libvirt-dev libcurl4-gnutls-dev dhcp3-server vblade apache2 unzip curl vlan  bridge-utils libvirt-bin sudo vtun (some of them may not be necessary for compute node installation, e.g., dhcp3-server, vtun, etc) 
    2020 1. get tarballs: 
    2121{{{ 
     
    2828   a. Axis2/C 
    2929   a. Rampart/C 
    30  1. useradd eucalyptus 
     30 1. # useradd eucalyptus 
    3131 1. run virsh list as eucaplytus, if it does not work - 
    32    a.  adduser eucalyptus libvirt 
     32   a.  # adduser eucalyptus libvirt 
    3333   a. edit /etc/xen/xend-config.sxp, make sure '(xend-unix-server yes)' and '/var/lib/xend/xend-socket' set, use lsof to verify xend-socket is open 
    3434 
     
    4545 1. Make sure the dataplane interface (although unconfigured) is UP 
    4646{{{ 
    47 $ sudo ifconfig eth0 up 
     47# ifconfig eth0 up 
    4848}}} 
    4949 1. [ALL] Test vconfig and brctl: 
     
    114114[ALL] 
    115115{{{ 
    116 /opt/eucalyptus/usr/sbin/euca_conf -d $EUCALYPTUS --hypervisor xen --instances /usr/local/eucalyptus --user eucalyptus --setup 
     116# /opt/eucalyptus/usr/sbin/euca_conf -d $EUCALYPTUS --hypervisor xen --instances /usr/local/eucalyptus --user eucalyptus --setup 
    117117}}} 
    118118 1. Euca startup scripts 
    119119[HN] 
    120120{{{ 
    121 ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud 
    122 ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc 
     121# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud 
     122# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc 
    123123}}} 
    124124 
    125125[CN] 
    126126{{{ 
    127 ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc 
     127# ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc 
    128128}}} 
    129129 
     
    132132 1. [HN]  
    133133{{{ 
    134 $EUCALYPTUS/usr/sbin/euca_conf --enable cloud --enable walrus --enable sc 
    135 $ /etc/init.d/eucalyptus-cloud restart 
    136 $ /etc/init.d/eucalyptus-cc restart 
     134# $EUCALYPTUS/usr/sbin/euca_conf --enable cloud --enable walrus --enable sc 
     135# /etc/init.d/eucalyptus-cloud restart 
     136# /etc/init.d/eucalyptus-cc restart 
    137137}}} 
    138138Note: eucalyptus-cc supports cleanstart/cleanstop/cleanrestart, when you change the conf file, you may need to do a clean restart. 
    139139 1. [CN] 
    140140{{{ 
    141 $ /etc/init.d/eucalyptus-nc restart 
     141# /etc/init.d/eucalyptus-nc restart 
    142142}}} 
    143143 1 
     
    166166 1. install python stuff 
    167167{{{ 
    168 apt-get install python-dev swig help2man libssl-dev 
     168# apt-get install python-dev swig help2man libssl-dev 
    169169}}} 
    170170 1. install boto 
     
    253253 1. Check if the bridges are created (eucabrXX) in the compute nodes: 
    254254{{{ 
    255 $ brctl show 
     255# brctl show 
    256256}}} 
    257257If something wrong, this procedure can be done manually so you can check the correctness of every step: 
    258258{{{ 
    259 $ sudo vconfig add eth0 10 
    260 $ sudo ifconfig eth0.10 up 
    261 $ sudo brctl addbr testbr10  
    262 $ sudo brctl addif testbr10 eth0.10 
    263 $ sudo ifconfig testbr10 10.0.0.2 up 
     259# vconfig add eth0 10 
     260# ifconfig eth0.10 up 
     261# brctl addbr testbr10  
     262# brctl addif testbr10 eth0.10 
     263# ifconfig testbr10 10.0.0.2 up 
    264264}}} 
    265265Do the same on front-end and make testbr10 10.0.0.1, make sure it can ping 10.0.0.2. Then undo what we just did for both front-end and the node: 
    266266{{{ 
    267 $ sudo ifconfig testbr10 down 
    268 $ sudo ifconfig eth0.10 down 
    269 $ sudo brctl delbr testbr10 
    270 $ sudo vonfig rem eth0.10 
     267# ifconfig testbr10 down 
     268# ifconfig eth0.10 down 
     269# brctl delbr testbr10 
     270# vonfig rem eth0.10 
    271271}}} 
    272272 1. run 
     
    277277 1. Check the VMs are created in machines (may have to hunt for them since you don't know which specific compute node a VM will be created on): 
    278278{{{ 
    279 $ virsh list 
     279# virsh list 
    280280}}} 
    281281You can use  
     
    288288 * [CN] /var/log/eucalyptus/nc.log 
    289289 
    290 == Modifying networking setup to work with ORCA == 
    291  
    292 In order to use Eucalyptus with ORCA each physical host must have two interfaces: one to the switch that is the dataplane (Cisco 6509 in RENCI's case) and one that leads either to a management network or to the public internet, to allow connection with ORCA actors. ORCA site authority for Euca will be deployed on the Eucalyptus master node and it must have 
    293  1. connectivity to other ORCA actors 
    294  1. connectivity to Euca slivers so it can install guests 
    295  
    296 [[Image(RENCI-Euca.png)]] 
    297  
    298 This is achieved by creating a bridge on each node with a known name. This example uses 'sliverbr' although the name is not important, as it is not known to ORCA and is hard-wired into Eucalyptus through a patch. The following procedure must be performed on each node (master and client). This presumes the eth1 on the physical host is the interface that leads into the management network or to the public internet. It must not have a configured IP address. It can be an 802.1q VLAN interface. 
    299  
    300  1. Create a bridge and add eth1 into it, then configure the bridge to be the default interface 
    301 {{{ 
    302 $ brctl add sliverbr 
    303 $ brctl addif sliverbr eth1 
    304 $ ifconfig sliverbr <public or management IP address> netmask <netmask>  
    305 $ route add default gw <default gw via the bridge interface> 
    306 }}} 
    307  
    308 In Ubuntu this can be accomplished by replacing eth1 configuration in /etc/network/interfaces file with the following: 
    309 {{{ 
    310 auto sliverbr 
    311 iface sliverbr inet static 
    312         bridge_ports eth1 
    313         bridge_stp off 
    314         bridge_maxwait 0 
    315         address <ip address> 
    316         netmask <netmak> 
    317         gateway <gateway> 
    318 }}} 
    319  
    320 and rebooting. 
    321  
    322 == Install Eucalyptus on master node from source == 
    323 Now that everything is working it is time to re-install the Eucalyptus master from source. Download the source code for 1.5.2 and follow the build instructions. It is advisable to build it in $EUCALYPTUS=/opt/eucalyptus to keep it out of the way of a packaged install. Pay attention to dependencies required to build it. Once built, install it, restart it and test access to the portal, then VM creation again. You can reuse the configuration file from the stock install by moving it to $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf.  
    324  
    325 Note that this procedure invalidates any previous configuration you had, so you have to establish new user credentials and upload new images from which VMs are created. 
    326  
    327 On Ubuntu 9.04 we had an issue with stock DHCP server that would not start properly after installing Eucalyptus master from source. It manifested itself by VMs being unreachable (in 'running' state). Log inspection (cc.log on master) revealed that dhcpd would not start when required. Our solution was to build a [http://www.isc.org/software/dhcp/313/download/dhcp-313targz dhcp server] from source and install it in a different location from the stock dhcpd. Then eucalyptus.conf had to be modified to reflect the new location of dhcpd.   
    328  
    329 == Installing ORCA-related patches on master node == 
    330  
    331 There are two patches - one for the VM creation template (to allow creation of VMs with more than one interface), the other to enable to specify the VLAN tag to be used for a particular security group.  
    332  1. Install the updated VM creation template on client nodes by replacing files gen_kvm_libvirt_xml  and gen_libvirt_xml in Eucalyptus. In Ubuntu/Debian they can be found under $EUCALYPTUS/usr/share/eucalyptus. The two files are attached to this page. 
    333  1. Install the patch (vlan.patch attached to this page) for Eucalyptus security group VLAN forcing on master node. Note that the user doing make and make install must have $JAVA_HOME, $EUCALYPTUS and $EUCALYPTUS_SRC defined and ant and java executables must be on the $PATH. 
    334 {{{ 
    335 $ cd eucalyptus-1.5.2/clc 
    336 $ patch -p2 < vlan.patch 
    337 $ make; make install 
    338 }}} 
    339  Restart the cloud controller and the portal, try the following as a regular user: 
    340 {{{ 
    341 $ euca-add-group -d testvlan vlan22 
    342 $ euca-run-instances -g vlan22 <usual parameters from above> 
    343 }}} 
    344 If this works, you should see that 'eucabr22' bridge has been created on every host and a 802.1q tagged interface (typycally eth0.22) was created and is part of that bridge. If VLAN id 22 is enabled on the switch between all hosts, then you should be able to reach the new VM on the IP address indicated by Eucalyptus and it will be on the private VLAN 22. 
    345  
    346 == Configuring ORCA to control the Eucalyptus cluster == 
    347  
    348 ORCA site authority must run from a container running on the Euca master node (otherwise the site authority has no access to the newly created VMs). Stand up an ORCA container with at least the Euca site authority. Here is the relevant sample piece of the actor_configs/config.xml: 
    349 {{{ 
    350                 <actor> 
    351                         <type>site</type> 
    352                         <name>duke-vm-site</name> 
    353                         <guid>9b12d036-23e7-11df-b3a3-000c29b1c193</guid> 
    354                         <pools> 
    355                                 <pool> 
    356                                         <type>duke.vm</type> 
    357                                         <label>Eucalyptus Virtual Machine (DUKE)</label> 
    358                                         <description>A virtual machine</description> 
    359                                         <units>10</units> 
    360                                         <start>2010-01-30T00:00:00</start> 
    361                                         <end>2011-01-30T00:00:00</end> 
    362                                         <handler path="ec2/handler.xml" /> 
    363                                         <attributes> 
    364                                                 <attribute> 
    365                                                         <key>resource.memory</key> 
    366                                                         <label>Memory</label> 
    367                                                         <value>128</value> 
    368                                                         <unit>MB</unit> 
    369                                                         <type>integer</type> 
    370                                                 </attribute> 
    371                                                 <attribute> 
    372                                                         <key>resource.cpu</key> 
    373                                                         <label>CPU</label> 
    374                                                         <value>1/2 of 2GHz Intel Xeon</value> 
    375                                                         <type>String</type> 
    376                                                 </attribute> 
    377                                         </attributes> 
    378                                         <properties> 
    379                                                 <property name="ip.list" value="192.168.206.3/24" /> 
    380                                                 <property name="ip.subnet" value="255.255.255.0" /> 
    381                                                 <property name="ip.gateway" value="192.168.206.1" /> 
    382                                                 <property name="data.subnet" value="255.255.0.0" /> 
    383                                         </properties> 
    384                                 </pool> 
    385                         </pools> 
    386                         <controls> 
    387                                 <control type="duke.vm" class="orca.policy.core.SimpleVMControl" /> 
    388                         </controls> 
    389                 </actor> 
    390 }}} 
    391  
    392 Note that this presumes an install where $ORCA_HOME contains the configuration files and they are not packaged in the webapp. 
    393  
    394 Once the container is up and running, you need to acquire credentials for ORCA from Eucalyptus. Login to the Eucalyptus portal, create a user for ORCA, export its credentials, which come in a zip file. 
    395  
    396 First test the credentials by unzipping them into $HOME/.euca, sourcing the .euca/XXX/eucarc file and making sure you can communicate with Eucalyptus using euca- tools. Create a keypair that ORCA will use (euca-add-keypair).  
    397  
    398 Now place the contents of the zip file under $ORCA_HOME/ec2 on the head node. Note that the zip file has a structure to it, which needs to be ignored. Simply copy the files from the lowest level of the zip file hierarchy into the $ORCA_HOME/ec2. Copy the generated ssh key (from euca-add-keypair) into the same directory. Modify the $ORCA_HOME/ec2/eucarc file as follows: 
    399  
    400 {{{ 
    401 #EUCA_KEY_DIR=$(dirname $(readlink -f ${BASH_SOURCE})) 
    402 export AMI_NAME=emi-6E7412EE 
    403 export EC2_SSH_KEY=orca-key-renci 
    404 export EC2_INSTANCE_TYPE=m1.small 
    405 }}} 
    406  
    407 (comment out the first line, add $AMI_NAME - the image to be used, $EC2_SSH_KEY and $EC2_INSTANCE_TYPE for ORCA to use). Note that AMI_NAME must have a default kernel and initrd image associated with it in Eucalyptus - they are currently not specified explicitly. 
    408  
    409 NOTE: For Bella 2.0 ORCA Euca authority logs into the VM, turns off DHCP and installs  BEN DNS server into /etc/resolv.conf. This may need to be modified in handlers/ec2/resources/scripts/prepare-net.sh 
    410  
    411 == Running Eucalyptus/EC2 handler tests == 
    412  
    413 = Undoing a packaged install = 
    414  
    415 When things don't seem to work, fear not, there is a way to start from scratch (note this is ONLY for DEB packaged installs, not installs from source): 
    416  
    417  1. Stop the euca daemons: 
    418 [HN] 
    419 {{{ 
    420 $ /etc/init.d/eucalyptus-cc stop 
    421 $ /etc/init.d/eucalyptus-cloud stop 
    422 }}} 
    423 [CN] 
    424 {{{ 
    425 $ /etc/init.d/eucalyptus-nc stop 
    426 }}} 
    427  1. Remove eucalyptus packages (including config directories, if possible) 
    428 [HN] 
    429 {{{ 
    430 $ dpkg --purge eucalyptus-cloud 
    431 $ dpkg --purge eucalyptus-cc 
    432 $ dpkg --purge eucalyptus-gl 
    433 $ dpkg --purge eucalyptus-common 
    434 $ dpkg --purge eucalyptus-javadeps 
    435 }}} 
    436 [CN] 
    437 {{{ 
    438 $ dpkg --purge eucalyptus-nc 
    439 $ dpkg --purge eucalyptus-gl 
    440 $ dpkg --purge eucalyptus-common 
    441 }}} 
    442  1. Remove user eucalyptus from the system 
    443 {{{ 
    444 $ userdel -r eucalyptus 
    445 $ groupdel eucalyptus 
    446 }}} 
    447  1. Remove remnants of config and log directories 
    448 {{{ 
    449 $ rm -rf /etc/eucalyptus 
    450 $ rm -rf /var/log/eucalyptus 
    451 }}} 
    452  1. Sometimes you may need to fix dpkg state 
    453 {{{  
    454 $ vi /var/lib/dpkg/statoverride 
    455 }}}  
    456 and remove the line that mentions 'eucalyptus' 
    457  1. Start over 
    458  
    459 = References =  
    460  
    461 [http://open.eucalyptus.com/wiki/EucalyptusInstallationUbuntuJaunty_v1.5.2 Euca install on Jaunty] 
    462  
    463 [https://bugs.launchpad.net/eucalyptus/+bug/417217 euca-group-add bug] 
    464  
    465 [http://open.eucalyptus.com/wiki/EucalyptusGettingStarted_v1.5.2 Getting started with Eucalyptus]