Installation of Kofa on Linux production system

Note

waeup.kofa and its custom packages might not work with Python > 2.7 currently. Use of Python 2.7 is recommended. The installation is described for Linux-based computers.

Part 1: Deploying Kofa as a single Zope client install

Prerequisites

The Kofa packages are based on Grok, which is a Python framework for agile webapplication development. Grok itself is based on Zope.

Both, Grok and Zope, are written in Python (with parts written in C). You therefore need Python installed.

Note, that you also need the Python header files and a compiler to compile the parts written in C.

To deploy Kofa most easily, we use zc.buildout

Preparing the system

To create a working copy of Kofa we recommend use of virtualenv. You, however, need also some basic libraries, a C compiler and some things more.

What you need (Debian/Ubuntu package names in brackets):

  • Python 2.7 (python2.7)

  • Python 2.7 development files (python2.7-dev)

  • A C-Compiler (gcc)

  • The C library development files (libc6-dev)

  • A subversion client (svn)

  • enscript (enscript) [optional]

    This is only needed if you want test coverage reports.

All these packages can be installed on Debian systems like this:

# apt-get install python2.7 python2.7-dev python2.7-dbg \
                  gcc libc6-dev svn enscript

Afterwards you should be able to enter:

$ python2.7

at the commandline and get a Python prompt. Quit the interpreter pressing <CTRL-D>.

Installing virtualenv

We recommend use of virtualenv to create Python sandboxes where you can run your code without touching any other installations.

If you don't already have easy_install available, you can find the script to set it up on the PEAK EasyInstall page.

You need to download ez_setup.py. Then, you run it like this to install easy_install into your system Python:

$ sudo python2.7 ez_setup.py

This will make easy_install available to you.

Note

Sometimes you have easy_install installed but you need a newer version of the underlying setuptools infrastructure to make Grok work. You can upgrade setuptools with:

$ sudo easy_install -U setuptools

Now you can install virtualenv by doing (as root):

# easy_install-2.7 virtualenv

This step will fetch all needed sources from the internet and install virtualenv locally in your Python2.7 installation.

Creating a sandbox

This step is only necessary (and recommended) if you installed virtualenv before.

As a normal user you now can create a sandbox for your upcoming work by:

$ virtualenv --no-site-packages mysandbox

where mysandbox is a directory in the filesystem where your sandbox will be created. virtualenv will also create this directory for you.

By passing the no-site-packages switch we tell virtualenv to provide us a clean environment without any extra-packages installed systemwide.

You now can activate the sandbox by doing:

$ source mysandbox/bin/activate

You will notice that the input prompt changes.

To deactivate the sandbox at any time, enter:

(sandbox27)$ deactivate

and the prompt will be the same as before the activation.

For the following steps make sure the sandbox is active.

Creating a working place

In the sandbox we now create our real working environment. To do this, we change to the sandbox and checkout the sources of Kofa from the subversion server:

(sandbox27)$ cd mysandbox/
(sandbox27)$ svn co https://svn.waeup.org/repos/main/waeup.kofa/trunk kofa-trunk

where kofa-trunk is only a name we've chosen here to make clear where the sources come from. In this case we are installing the Kofa base package.

The command should fetch the Kofa base package sources for you and put it in the directory kofa-trunk/.

Now enter the new directory:

(sandbox27)$ cd kofa-trunk

Preparing the build

In the sources directory (kofa-trunk/) you have to prepare the project to fetch needed components (eggs), compile C-code parts, etc. This steip will not touch any external projects:

(sandbox27)$ python bootstrap.py

This will generate some directories and the buildout script in bin/ for us. This step must be executed only once for each instance.

You can now deactivate the sandbox:

(sandbox27)$ deactivate

Now we can do the real build by triggering:

$ bin/buildout

If this is your first install of some Grok-related project, this step will need some time as lots of sources have to be fetched, many components must be compiled, etc.

This step must be redone whenever you change something in buildout.cfg or setup.py.

Note that if you have more than one sandbox for a Zope-based web application, it will probably make sense to share the eggs between the different sandboxes. You can tell zc.buildout to use a central eggs directory by creating ~/.buildout/default.cfg with the following contents:

[buildout]
eggs-directory = /home/bruno/buildout-eggs

Start the instance

You should be able now to start the created instance by doing:

$ bin/zopectl fg

Alternatively you can do:

$ bin/paster serve parts/etc/deploy.ini

The port numbers where Kofa is running on your server are defined in buildout.cfg under [kofa_params].

If you now point a browser to the right port on your server, for example

localhost:8080

you should get a login pop-up, where you can login as superuser with grok and grok as username/password (Kofa base package only!).

You can stop the instance by pressing <CTRL-C>.

If you are connected and logged in, you should be able to add the grok-based applications (such as University) from the menu.

Add an instance of University and click on the link next to the then visible entry in the list of installed applications.

Running the tests

The tests are easily run by executing the test runner that's installed in the bin directory:

$ bin/test

Part 2: Deploying Kofa as ZEO install

Each ZEO install consists of at least one ZEO server and normally two or more ZEO clients. While the ZEO server is meant to manage the ZODB database for clients, the clients connect to the outside world, listen for request and do the real dataprocessing.

We prepared a buildout configuration that sets up one server configuration and two client configs. This configuration is in buildout-zeo.cfg.

Generating the ZEO setup

To install Kofa ZEO-based you can run buildout with the given (or your own) configuration file like this:

$ ./bin/buildout -c buildout-zeo.cfg

This should generate all scripts necessary to run servers, clients, etc.

Starting ZEO servers and clients

First start the server:

$ ./bin/zeo_server start

Clients can be started by doing:

$ ./bin/zeo_client1 start
$ ./bin/zeo_client2 start

This will start both clients in daemon mode.

Instead of start you can, as usually, start an instance in foreground (fg), etc. You know the drill.

Manually starting ZEO clients

This is normally not neccessary.

zeo_clientN scripts are basically wrappers around calls to bin/paster. You can bypass this wrapper and start a client 'manually' like this:

$ ./bin/paster serve --pid-file var/zeo1.pid --daemon \
      pars/etc/zeo1.ini

It is important to give a pid-file as paster otherwise can not start different clients (they would all refer to the same pid file paster.pid and refuse to start after the first client was started).

Setup (paramters, ports, etc.)

By default the server will listen on port 8100 for requests from localhost (not: from the outside world).

You can change ZEO server settings in the [zeo_server] section of buildout-zeo.conf. Run buildout afterwards.

The clients will listen on port 8081 and 8082. You can change settings in etc/zeo1.ini.in and etc/zeo2.ini.in respectively. Run buildout after any change.

If you want to change the paster wrapper for any zeo client, you can edit etc/zeo1.conf and/or etc/zeo2.conf. Run buildout afterwards.

Creating new clients

You want more clients to be created by buildout? Easy. Three steps are neccessary.

1. Create config files in etc/

Each client needs two configuration files:

  • etc/zeoN.conf

    configuring the paster wrapper

  • etc/zeoN.ini

    configuring the runtime config, ports, etc.

Just copy over these files from the already existing zeo1/zeo2 files and replace zeo1 or zeo2 with your new name.

2. Update buildout-zeo.cfg

Here, inside buildourt-zeo.cfg also three steps are needed.

  • 2.1. Create new .ini and .conf entries

    The .conf and .ini files in etc/ are only templates that have to be generated in their really used final location. In buildout-zeo.cfg you can care for this by creating a new [zeoN_ini] and [zeoN_conf] option (replacing N with a number, of course).

    Just copy over existing entries and replace the mentions of zeo1 or zeo2 by your zeoN.

  • 2.2. Create a new zeo_clientN entry

    Then you have to create an entry that will generate the zeo_clientN script. Again, just copy over an existing [zeo_client1] entry and replace 1 withg your client number.

  • 2.3. Register the new sections in [buildout] section

    When done with the above: add the new section in [buildout]:

    [buildout]
      ...
      <old entries...>
      ...
      zope_conf_zeo_5
      zeo5_ini
      zeo_client5
    

    depending on how you named your new sections.

3. Rerun buildout

When adding or removing client/server instances, make sure to stop all running servers/clients before rerunning buildout.

To activate the new setup, rerun buildout:

$ bin/buildout -c buildout-zeo.cfg

This should generate any new clients and remove older ones or just update configuration files.

Considerations

There are some things in the current buildout-zeo.cfg we might do not want. It extends the regular buildout.cfg so that we do not have to repeat most sections but the parts in [buildout] have to be listed.

We need, however, not everything with a ZEO-deploy that is listed in a default buildout. We might do not need docs, no profiling, etc. Also a regular non-ZEO kofactl might not make to much sense. Therefore all this might be subject to changes.