Version 14 (modified by ibaldin, 9 years ago)

--

Best practices for setting up a distributed production environment

Introduction

Setting up a production ORCA configuration requires careful and meticulous approach to avoid common problems. This document attempts to summarize best practices used at RENCI for deploying ORCA.

Prerequisites

Software prerequisites

Build ORCA from source

Understanding container configuration: ORCA_HOME, ORCA_LOCAL, and all that

Actor configuration

Preparation

Preparing the infrastructure

  1. You must decide and document the topology of the ORCA actors in the future deployment - how many Tomcat containers you will have, which hosts they will be located on and which actors will be deployed in which container.
  2. Setup tomcat on each of the hosts. The canonical way, in which RENCI sets up ORCA is to have $ORCA_HOME=/opt/orca on each of the deployment hosts. ORCA-modified tomcat is installed under $ORCA_HOME/tomcat and $ORCA_HOME/tomcat/start.sh and $ORCA_HOME/tomcat/stop.sh scripts are modified to reflect the $ORCA_HOME setting by prepending them with
    export ORCA_HOME=/opt/orca
    
  3. MySQL database on each of the hosts must be initialized with the schema and initial data. Loading inventory files is optional.
  4. Verify that tomcat starts and stops properly without ORCA

Preparing the configuration

The deployment can use a single source tree located on the machine, where you build ORCA, which has Java, Ant and Maven. The hosts with Tomcat containers should have the same version of Java as is used on the build machine.

  1. Build ORCA as usual
    $ cd $ORCA_SRC
    $ mvn install
    
  2. Prepare the directory structure with configuration files on the build host. Basically for each host with a container you should have a separate copy of $ORCA_SRC/webapp (or $ORCA_SRC/webapp2, if present). One way to do this:
    $ cd $HOME
    $ mkdir host1 host2 host3
    $ cd $ORCA_SRC/webapp2
    $ tar -cf - . | tar -xf - -C $HOME/host1
    $ tar -cf - . | tar -xf - -C $HOME/host2
    $ tar -cf - . | tar -xf - -C $HOME/host3
    
  3. For each host, generate and write down a new container GUID. This can be done in a number of ways. Edit hostX/config/container.properties to replace the container.guid property with a new value.
  4. To make sure actors from this container are registered with ORCA actor registry, add or edit the following statements at the bottom of container.properties:
    registry.url=http://geni.renci.org:8080/registry/
    registry.method=registryService.insert
    
  5. Be sure the property values in container.properties for MySQL credentials on the host are valid
  6. Copy hostX/config/container.properties to $ORCA_HOME/config/ on hostX
  7. Generate a new security configuration (if not yet done) and a guid and certificate for each new actor (take note of the GUID)
    $ cd $ORCA_SRC/tools/config
    $ ant security.create.admin.config
    $ ant guid
    $ ant security.create.actor.config -Dactor=<Actor GUID>
    
  8. Copy the $ORCA_SRC/tools/config/runtime directory to $ORCA_HOME on the host where the actors will reside. In the instructions below if ORCA_HOME is not defined as an environment variable for the user, you have to replace it with an explicit path.
    $ cd $ORCA_SRC/tools/config
    $ tar -cf - runtime/ | ssh user@hostX tar -xf - -C $ORCA_HOME
    $ tar -cf - scripts/ | ssh user@hostX tar -xf - -C $ORCA_HOME
    

Deploying actors

  1. Create the actor configuration for the container by editing $HOME/hostX/actor_configs/config.xml for each host. At the bottom of this page is a working example of a configuration file.
  2. To lookup certificates for actors in other containers (if they are already running), consult ORCA actor registry. You can extract them from your own keystores as described here
  3. Edit the $HOME/hostX/ant/build.properties to point to the URL of the container on hostX
  4. Package and deploy
    $ cd $HOME/hostX
    $ mvn package
    $ ant deploy
    

Attachments