Version 41 (modified by zhaomuzh, 8 years ago)

--

Eucalyptus/XCat image proxy

Overview

ImageProxy is a mechanism used to make the same guest image available across multiple independent cloud sites. It downloads images named by URLs (including support for Bittorrent URLs) and registers them with the site's cloud provider (e.g. Eucalyptus) using local user credentials. ImageProxy is implemented as an Axis2 SOAP service that, when provided with an URL for an image and a site-unique GUID to name it, returns the EMI of the registered image. If the image is not present within the system, it is downloaded and registered; otherwise the EMI of the previously registered image is returned.

Installation

  • Download the source (latest tag or trunk). The environment variable IMAGEPROXY_SRC denotes the location of the source, in this document.
    $ svn co https://geni-orca.renci.org/svn/orca-external/imageproxy/trunk imageproxy
    $ cd imageproxy
    $ export IMAGEPROXY_SRC=`pwd`
    
  • Alter the properties file "$IMAGEPROXY_SRC/trunk/src/main/resources/orca/imageproxy/imageproxy-settings.properties" as desired for your site, e.g.
    # relative to installation directory IMAGEPROXY_HOME
    registerScriptProperty = scripts/register.sh
    # euca bukkit name
    eucaBukkitName = imageproxy.bukkit
    # testmode - use image url as EMI, do not perform any downloads
    impTestMode = false
    # how long to sleep for in test mode (to pretend to download an image) [sec]
    testModeSleep = 30
    # the total disk space to cache the images (the unit is GB, the default size is 1GB)
    spacesize = 10
    # the database file used to store the metadata of images
    db.imageproxy.db = imageproxy.db
    ...
    
    • Be sure to set impTestMode to "false" for a production environment.
    • Be sure to set eucaBukkitName to a unique, unused name, or to a bucket name "owned" by the Eucalyptus credentials you plan to use with ImageProxy?.
  • Define the following two environment variables as appropriate for your site.
    $ export IMAGEPROXY_HOME=/opt/imageproxy
    $ export PROXY_EUCA_KEY_DIR=$IMAGEPROXY_HOME/ec2
    
  • ImageProxy requires using an Axis2-capable servlet container. You can use the one that comes with the Axis2 distribution or you can add Axis2 support to Tomcat. Download Axis2 (either the binary distribution for stand-alone operation or WAR for running under Tomcat) from the Axis2 download site. At this time, we advise using a standalone Axis2 container.
    • Define AXIS2_HOME environment variable
      $ export AXIS2_HOME=/opt/imageproxy/axis-{version}
      
    • For standalone operation, unzip Axis2 zip into $IMAGEPROXY_HOME
      • Modify the Axis2 start script (axis2server.sh in $AXIS2_HOME/bin) to include the following environment variable definitions:
        export IMAGEPROXY_HOME="/opt/imageproxy"
        export LD_LIBRARY_PATH="$IMAGEPROXY_HOME/lib:$LD_LIBRARY_PATH"
        export PROXY_EUCA_KEY_DIR="$IMAGEPROXY_HOME/ec2"
        export IMAGEPROXY_LOG="$IMAGEPROXY_HOME/logs/register.log"
        # The following line is needed to ensure Axis2 does not run out of file descriptors.
        # Please ensure that your environment permits increasing this limit.
        ulimit -n 4096
        
    • For deployment under Tomcat, place the Axis2 war into the webapps directory in your Tomcat installation. Ensure that Tomcat is running, and test the installation by checking the following URL: http://$TOMCAT_HOST:$TOMCAT_PORT/axis2
      • Modify the Tomcat startup.sh script to include the following environment variable definitions:
        export IMAGEPROXY_HOME="/opt/imageproxy"
        export LD_LIBRARY_PATH="$IMAGEPROXY_HOME/lib:$LD_LIBRARY_PATH"
        export PROXY_EUCA_KEY_DIR="$IMAGEPROXY_HOME/ec2"
        export IMAGEPROXY_LOG="$IMAGEPROXY_HOME/logs/register.log"
        
  • Download and install Sqlite from the Sqlite download site
    • Test to check if installed
       $ sqlite3 -version
       3.7.5
      
  • Build ImageProxy components
    • Bittorrent JNI
      $ cd $IMAGEPROXY_SRC/bittorrent_jni
      $ ./prep_build.sh 
      $ ./configure --prefix=$IMAGEPROXY_HOME --with-jdk=/path/to/jdk
      $ make
      $ make install
      
      • NOTE1: prep_build.sh script attempts to download and patch some software components
      • NOTE2: configure step depends on having the libevent library and headers present in the system. If your system does not have libevent installed, it can be compiled and installed from source. Use the latest libevent from 1.4 series. To point configure step at the location of libevent headers and binaries, you can use CFLAGS and LDFLAGS environment variables.
      • NOTE3: Additional library and header dependencies checked for by configure are: OpenSSL, zlib, and libCURL.
      • NOTE4: For Centos systems, link the libtool in $IMAGEPROXY_SRC to the system libtool (/usr/bin/libtool)
    • Compiling the SOAP webservice
      $ cd $IMAGEPROXY_SRC
      $ mvn install axis2-aar:aar
      
  • Install Eucalyptus credentials under $PROXY_EUCA_KEY_DIR by unzipping the Eucalyptus credentials archive in that directory
  • Install the scripts under $IMAGEPROXY_HOME/scripts (the location of the scripts can be adjusted in imageproxy-settings.properties file prior to compilation and deployment, however the path is always relative to $IMAGEPROXY_HOME).
    $ mkdir $IMAGEPROXY_HOME/scripts
    $ cp $IMAGEPROXY_SRC/scripts/* $IMAGEPROXY_HOME/scripts
    
    • NOTE1: mkdir $IMAGEPROXY_HOME/logs; mkdir $IMAGEPROXY_HOME/settings; if they don't exist
  • Deploy the service
    • For standalone Axis2, copy the jar file into the services directory and start the service
      $ cp target/imageproxy-1.0-SNAPSHOT.jar $AXIS2_HOME/repository/services/
      $ cd $AXIS2_HOME; nohup bin/axis2server.sh &
      
    • For running under Tomcat, copy the aar file into the services directory and restart Tomcat
      $ cp target/imageproxy-1.0-SNAPSHOT.aar $TOMCAT_ROOT/webapps/axis2/WEB-INF/services
      $ $TOMCAT_ROOT/shutdown.sh; $TOMCAT_ROOT/startup.sh
      
  • Start the seeding process for Bittorrent downloads:
    $ cd $IMAGEPROXY_HOME
    $ nohup ./bin/seeding $IMAGEPROXY_HOME ${location of database file} > $IMAGEPROXY_HOME/logs/seeding.log &
    

Testing

  • Ensure that the IMAGEPROXY service is present by visiting the appropriate URL.
  • Run the test client to register an image. A brief usage description can be obtained by running:
    $ $IMAGEPROXY_HOME/scripts/run.sh --help
    
  • NOTE: ImageProxy supports a test mode that does not download or register images but returns consistent EMIs for URLs and guids. See imageproxy-settings.properties. Service needs to be recompiled (mvn install axis2-aar:aar) and reinstalled to take effect.

Removing imageproxy-created images from Euca

$ for emi in `euca-describe-images  | grep imageproxy | awk '{ print $2}'`; do echo $emi; euca-deregister $emi; done
$ euca-delete-bundle -b imageproxy.bukkit