source: main/waeup.sirp/branches/henrik-bootstrap/docs/source/developerdocs/developernotes.rst @ 11110

Developer Notes
***************

How to setup a developer instance of the WAeUP SIRP, handle tests, docs
etc.

The new WAeUP SIRP is based on `Grok <http://grok.zope.org/>`_.

Installing a developer copy
===========================

The installation is described for Linux-based computers.

Preparing the system
--------------------

To create a working copy of the WAeUP SIRP we recommend use of
`virtualenv`. You, however, need also some basic libraries, a C
compiler and some things more.

What you need (Debian/Ubuntu package names in brackets):

* Python 2.6 (python2.6)

  Currently, also Python2.5 is supported but we want to make use of
  some of the 2.6 goodies in the future.


* Python 2.6 development files (python2.6-dev)

* A C-Compiler (gcc)

* The C library development files (libc6-dev)

* A subversion client (svn)

* enscript (enscript) [optional]

  This is only needed if you want test coverage reports.

All these packages can be installed on Debian systems like this::

  # apt-get install python2.6 python2.6-dev python2.6-dbg \
                    gcc libc6-dev svn enscript

Afterwards you should be able to enter::

  $ python2.6

at the commandline and get a Python prompt. Quit the interpreter
pressing <CTRL-D>.

Installing `virtualenv`
-----------------------

We recommend use of `virtualenv` to create Python sandboxes where you
can run your code without touching any other installations.

If you don't already have ``easy_install`` available, you can find the
script to set it up on the `PEAK EasyInstall page`_.

.. _`PEAK EasyInstall page`: http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install

You need to download `ez_setup.py`_. Then, you run it like this to
install ``easy_install`` into your system Python::

  $ sudo python2.6 ez_setup.py

.. _`ez_setup.py`: http://peak.telecommunity.com/dist/ez_setup.py

This will make ``easy_install`` available to you.

.. note:: Sometimes you have ``easy_install`` installed but you need a
          newer version of the underlying setuptools infrastructure to
          make Grok work. You can upgrade setuptools with::

            $ sudo easy_install -U setuptools

Now you can install `virtualenv` by doing (as root)::

  # easy_install-2.6 virtualenv

This step will fetch all needed sources from the internet and install
`virtualenv` locally in your Python2.6 installation.


Creating a sandbox
------------------

This step is only necessary (and recommended) if you installed
`virtualenv` before.

As a normal user you now can create a sandbox for your upcoming work
by::

  $ virtualenv --no-site-packages mysandbox

where ``mysandbox`` is a directory in the filesystem where your
sandbox will be created. `virtualenv` will also create this directory
for you.

By passing the ``no-site-packages`` switch we tell `virtualenv` to
provide us a clean environment without any extra-packages installed
systemwide.

If you have a look into the freshly created sandbox, you will notice
that in the ``bin/`` directory there is also 

You now can activate the sandbox by doing::

  $ source mysandbox/bin/activate

You will notice that the input prompt changes.

To deactivate the sandbox at any time, enter::

  $ deactivate

and the prompt will be the same as before the activation.

For the following steps make sure the sandbox is active.


Creating a working place
------------------------

In the sandbox (or anywhere else) we now create our real working
environment. To do this, we change to the sandbox and checkout the
sources of the WAeUP SIRP from the subversion server::

  $ cd mysandbox/
  $ svn co https://svn.waeup.org/repos/main/waeup.sirp/trunk waeup-trunk

where ``waeup-trunk`` is only a name we've chosen here to make clear
where the sources come from.

This command should fetch the sources of the WAeUP sources for you and
put it in the directory ``waeup-trunk/``.

Now enter the new directory::

  $ cd waeup-trunk


Preparing the build
-------------------

In the sources directory (``waeup-trunk/``) you have to prepare the
project to fetch needed components (eggs), compile C-code parts,
etc. This steip will not touch any external projects::

  $ python2.6 bootstrap.py

This will generate some directories and the ``buildout`` script in
``bin/`` for us. This step must be executed only once for each
instance.

Now we can do the real build by triggering::

  $ bin/buildout

If this is your first install of some Grok-related project, this step
will need some time as lots of sources have to be fetched, many
components must be compiled, etc.

This step must be redone whenever you change something in
``buildout.cfg`` or ``setup.py``.

Afterwards we are ready to go.


Start the instance
------------------

You should be able now to start the created instance by doing::

   $ bin/zopectl fg

If you now point a browser to::

   localhost:8080

you should get a login pop-up, where you can login as superuser with
``grok`` and ``grok`` as username/password.

If you want to change the default credentials, have a look into
``buildout.cfg`` where the superuser password is determined.

You can stop the instance by pressing <CTRL-C>.



Documentation
=============

With the :mod:`waeup.sirp` package we try to reach high standards in
both, documentation and testing.

:mod:`waeup.sirp` makes extensive use of doctests, which this way also
become both: executable (i.e. testable) examples and documentation.

Generating documentation
------------------------

We use the excellent `Sphinx <http://sphinx.pocoo.org/>`_ Python
documentation generator to generate the docs as HTML pages.

The documentation of the :mod:`waeup.sirp` project can easily be
created doing::

  $ bin/waeupdocs

This will create a tree of HTML pages in
``parts/waeupdocs/waeup.sirp/build/waeup.sirp/`` which you can for
instance browse by pointing your browser to this location.

An 'official' place in internet for the whole docs is about to come
but not yet available.


Writing documentation
---------------------

means explaining to other developers what your code does and test it
at the same time. See the many .txt files in the :mod:`waeup.sirp`
package for examples.


Testing
=======

Tests are most important to the reliability of the :mod:`waeup.sirp`
package. We don't tell someone that our code works, if we cannot prove
it. And we prove it by testing.

Running tests
-------------

You can run all the tests for the package by doing::

  $ bin/test

If you like colored output (can improve readability of failure
messages), use the ``-c`` switch::

  $ bin/test -c

We have many tests in the :mod:`waeup.sirp` package so that sometimes
you only want the functional *or* the unit tests to run. This can be
done like this::

  $ bin/test -c -u

if you want only unit tests or like this::

  $ bin/test -c -f

if you only want functional tests.


Writing tests
-------------

Is a wiiide topic. For now, please see the numerous .txt files in the
source. Most of the **are** tests.

Generally, we use `z3c.testsetup` for finding and setting up tests.

There are mainly two kinds of tests we use:

* unit- or simple doctests

  These are included in test runs automatically, if they provide a
  marker like this somewhere (normally near top of file)::

    .. :doctest:

  Most unit tests furthermore declare that they want to be run inside
  the `WAeUPSIRPUnitTestLayer` defined in `waeup.sirp.testing`. This
  layer groks the whole `waeup.sirp` package, so that all ZCA
  components are already setup when you start your tests.
  
  To declare that a unit test testfile should be run inside this
  layer, the testfile has to provide the following line::

    .. :layer: waeup.sirp.testing.WAeUPSIRPUnitTestLayer

  Use it, if in your tests you make use of registered components like
  utilities, adapters and the like.

* integration or functional tests

  These provide a full-blown ZODB storage, so that we can emulate
  browser requests to the whole system. Functional tests are much more
  expensive in terms of memory and runtime but needed, if you want to
  test UI components.

  A testfile is registered as functional test on testruns when it
  provides a line like the following::

    :Test-Layer: functional


Code coverage
-------------

We want to make sure, that all aspects of our software are
tested. This means, that all parts of the codes should be tested
somewhere.

To tell how good our test coverage is, we can also use the testrunner
(``bin/test``)::

  $ bin/test --coverage=coverage

will run the tests but also look, which parts of code were touched by
them. For releases we want 100% coverage. Beware: running tests with
the ``--coverage`` switch slows down tests by factor 10 or more.

The command above will output a table with percentages. Furthermore,
in ``/parts/test/coverage`` you will (after the testrun) find your
sources preceeded by markers which tell, how often (or none) a certain
line was used in tests.

To have a more convenient cmdline interface, we also provide some
shortcuts::

  $ bin/coverage-detect

will run all the tests as shown above and put the results in
``parts/coverage-detect/coverage``.

After that you can run::

  $ bin/coveragereport

to get a browsable HTML representation of test coverage in
``coverage-report/`` subdir.

Both, the coverage reports and HTML documentation generated by sphinx
can be packed and put onto a website as-is.

It is also possible to generate the docs and reports nightly by a
buildbot or something like this.