Version 36 (modified by ibaldin, 8 years ago)


Best practices for setting up a distributed production environment


Setting up a production ORCA configuration requires careful and meticulous approach to avoid common problems. This document attempts to summarize best practices used at RENCI for deploying ORCA.


Software prerequisites

Build ORCA from source

Understanding container configuration: ORCA_HOME, ORCA_LOCAL, and all that

Actor configuration


Preparing the infrastructure

  1. You must decide and document the topology of the ORCA actors in the future deployment - how many Tomcat containers you will have, which hosts they will be located on and which actors will be deployed in which container.
    • BIG FAT NOTE: current design prevents a service manager from talking to remote authorities if it is co-located with the broker in the same container. If your service manager is expected to redeem on remote sites, do not put it in the same container as the broker.
  2. Setup ORCA Configuration directories on each host
  3. Setup tomcat on each of the hosts.
  4. Setup MySQL database on each of the hosts
  5. Verify that tomcat starts and stops properly without ORCA
    $ cd $ORCA_HOME/tomcat
    $ ./
    $ ./

Preparing the configuration and deploying

You can deploy from a binary release or from source. If deploying from source the deployment can use a single source tree located on the machine, where you build ORCA, which has Java, Ant and Maven. The hosts with Tomcat containers should have the same version of Java as is used on the build machine.

Bella 2.2

  1. Build ORCA as usual
    $ cd $ORCA_SRC
    $ mvn install
  2. Prepare the directory structure with configuration files on the build host. Basically for each host with a container you should have a separate copy of $ORCA_SRC/webapp (or $ORCA_SRC/webapp2, if present). One way to do this:
    $ cd $HOME
    $ mkdir host1 host2 host3
    $ cd $ORCA_SRC/webapp2
    $ tar -cf - . | tar -xf - -C $HOME/host1
    $ tar -cf - . | tar -xf - -C $HOME/host2
    $ tar -cf - . | tar -xf - -C $HOME/host3
  3. For each host, generate and write down a new container GUID. This can be done in a number of ways.
  4. Edit hostX/config/
    1. replace the container.guid property with a new value.
    2. Update the container URL property to the host on which it will be deployed.
    3. Be sure to set emulation=false
    4. To make sure actors from this container are registered with ORCA actor registry, add or edit the following statements at the bottom of
    5. Be sure the property values in for MySQL credentials on the host are valid
  5. Copy hostX/config/ to $ORCA_HOME/config/ on hostX
  6. Generate a new security configuration (if not yet done) and a guid and certificate for each new actor (take note of the GUID)
    $ cd $ORCA_SRC/tools/config
    $ ant security.create.admin.config (THIS ONLY NEEDS TO BE DONE ONCE!!!)
    $ ant guid
    $ ant -Dactor=<Actor GUID>
  7. Copy the $ORCA_SRC/tools/config/runtime directory to $ORCA_HOME on the host where the actors will reside. In the instructions below if ORCA_HOME is not defined as an environment variable for the user, you have to replace it with an explicit path.
    $ cd $ORCA_SRC/tools/config
    $ tar -cf - runtime/ | ssh user@hostX tar -xf - -C $ORCA_HOME
    $ tar -cf - scripts/ | ssh user@hostX tar -xf - -C $ORCA_HOME
    1. CAVEAT: this method obviously accumulates certificates of all actors across containers. This is OK for some definition of OK. The important thing is that the container has a version of the keystore that contains the certificates for actors in that container. Any extra certificates will not harm but present a potential security loophole. A more involved method would create separate runtime/ directories for each container.
  8. Create the actor configuration for the container by editing $HOME/hostX/actor_configs/config.xml for each host. You will need the GUIDs you have issued to the actors. At the bottom of this page is a working example of a configuration file.
  9. To lookup certificates for actors in other containers (if they are already running), consult ORCA actor registry. You can extract them from your own keystores as described here
  10. Edit the $HOME/hostX/ant/ to point to the URL of the container on hostX
  11. Package and deploy for each host (note that Tomcat must be running for the deploy step too succeed as it uses Tomcat Manager webapp to remotely install ORCA webapp into Tomcat).
    $ cd $HOME/hostX
    $ mvn package
    $ ant deploy
  12. Check that your actors are properly registered by visiting the ORCA actor registry.

Camano 3.0

  1. Generate GUIDs and certificates for all actors, copy the contents of $ORCA_SRC/tools/config/runtime to each host's $ORCA_HOME/config/runtime
  2. Create $ORCA_HOME/config/ and $ORCA_HOME/config/config.xml actor configuration file for each host
  3. Add any additional configuration, like e.g. for an authority actor
  4. Check that your actors are properly registered by visiting the ORCA actor registry.