Index: /main/waeup.custom/trunk/README-zeo.txt
===================================================================
--- /main/waeup.custom/trunk/README-zeo.txt	(revision 8036)
+++ /main/waeup.custom/trunk/README-zeo.txt	(revision 8036)
@@ -0,0 +1,156 @@
+Deploying waeup.custom as ZEO-install
+*************************************
+
+Each ZEO install consists of at least one ZEO server and normally two
+or more ZEO clients. While the ZEO server is meant to manage the ZODB
+database for clients, the clients connect to the outside world, listen
+for request and do the real dataprocessing.
+
+We prepared a `buildout` configuration that sets up one server
+configuration and two client configs. This configuration is in
+``buildout-zeo.cfg``.
+
+Generating the ZEO setup
+========================
+
+To install waeup.custom ZEO-based you can run `buildout` with the given
+(or your own) configuration file like this::
+
+  $ ./bin/buildout -c buildout-zeo.cfg
+
+This should generate all scripts necessary to run servers, clients,
+etc.
+
+Starting ZEO servers and clients
+================================
+
+First start the server::
+
+  $ ./bin/zeo_server start
+
+Clients can be started by doing::
+
+  $ ./bin/zeo_client1 start
+  $ ./bin/zeo_client2 start
+
+This will start both clients in daemon mode.
+
+Instead of ``start`` you can, as usually, start an instance in
+foreground (``fg``), etc. You know the drill.
+
+Manually starting ZEO clients
+-----------------------------
+
+This is normally not neccessary.
+
+``zeo_clientN`` scripts are basically wrappers around calls to
+``bin/paster``. You can bypass this wrapper and start a client
+'manually' like this::
+
+  $ ./bin/paster serve --pid-file var/zeo1.pid --daemon \
+        pars/etc/zeo1.ini
+
+It is important to give a pid-file as paster otherwise can not start
+different clients (they would all refer to the same pid file
+`paster.pid` and refuse to start after the first client was started).
+
+
+Setup (paramters, ports, etc.)
+==============================
+
+By default the server will listen on port 8100 for requests from
+localhost (not: from the outside world).
+
+You can change ZEO server settings in the ``[zeo_server]`` section
+of ``buildout-zeo.conf``. Run `buildout` afterwards.
+
+The clients will listen on port 8081 and 8082. You can change settings
+in ``etc/zeo1.ini.in`` and ``etc/zeo2.ini.in`` respectively. Run
+buildout after any change.
+
+If you want to change the paster wrapper for any zeo client, you can
+edit ``etc/zeo1.conf`` and/or ``etc/zeo2.conf``. Run buildout
+afterwards.
+
+Creating new clients
+====================
+
+You want more clients to be created by buildout? Easy. Three steps are
+neccessary.
+
+1. Create config files in etc/
+------------------------------
+
+Each client needs two configuration files:
+
+  - ``etc/zeoN.conf``
+       configuring the paster wrapper
+
+  - ``etc/zeoN.ini``
+       configuring the runtime config, ports, etc.
+
+Just copy over these files from the already existing zeo1/zeo2 files
+and replace ``zeo1`` or ``zeo2`` with your new name.
+
+2. Update buildout-zeo.cfg
+--------------------------
+
+Here, inside ``buildourt-zeo.cfg`` also three steps are needed.
+
+* 2.1. Create new .ini and .conf entries
+
+  The .conf and .ini files in etc/ are only templates that have to be
+  generated in their really used final location. In buildout-zeo.cfg
+  you can care for this by creating a new ``[zeoN_ini]`` and
+  ``[zeoN_conf]`` option (replacing ``N`` with a number, of course).
+
+  Just copy over existing entries and replace the mentions of ``zeo1``
+  or ``zeo2`` by your ``zeoN``.
+
+* 2.2. Create a new ``zeo_clientN`` entry
+
+  Then you have to create an entry that will generate the
+  ``zeo_clientN`` script. Again, just copy over an existing
+  ``[zeo_client1]`` entry and replace ``1`` withg your client number.
+
+* 2.3. Register the new sections in ``[buildout]`` section
+
+  When done with the above: add the new section in ``[buildout]``::
+
+    [buildout]
+      ...
+      <old entries...>
+      ...
+      zope_conf_zeo_5
+      zeo5_ini
+      zeo_client5
+
+  depending on how you named your new sections.
+
+3. Rerun ``buildout``
+---------------------
+
+When adding or removing client/server instances, make sure to stop all
+running servers/clients before rerunning buildout.
+
+To activate the new setup, rerun buildout::
+
+  $ bin/buildout -c buildout-zeo.cfg
+
+This should generate any new clients and remove older ones or just
+update configuration files.
+
+
+Considerations
+==============
+
+There are some things in the current buildout-zeo.cfg we might do not
+want. It extends the regular ``buildout.cfg`` so that we do not have
+to repeat most sections but the ``parts`` in ``[buildout]`` have to be
+listed.
+
+We need, however, not everything with a ZEO-deploy that is listed in a
+default buildout. We might do not need docs, no profiling, etc. Also a
+regular non-ZEO kofactl might not make to much sense. Therefore all
+this might be subject to changes.
+
Index: /main/waeup.custom/trunk/buildout-zeo.cfg
===================================================================
--- /main/waeup.custom/trunk/buildout-zeo.cfg	(revision 8036)
+++ /main/waeup.custom/trunk/buildout-zeo.cfg	(revision 8036)
@@ -0,0 +1,124 @@
+# A buildout configuration for a ZEO server and two ZEO clients.
+#
+[buildout]
+extends = buildout.cfg
+parts =
+    eggbasket
+    app
+    i18n
+    test
+    mkdirs
+    zpasswd
+    zope_conf
+    site_zcml
+    zdaemon_conf
+    deploy_ini
+    debug_ini
+    profile_ini
+    zeo1_ini
+    zeo2_ini
+    waeupdocs
+    coverage-detect
+    coverage-report
+# kofactl creation must be _after_ app!
+    kofactl
+# For backward compatibility, telling buildout not to throw away
+# the data and log subdirectories from the parts directory.
+    data
+    log
+
+    zeo_server
+    zope_conf_zeo_1
+    zope_conf_zeo_2
+    zeo1_ini
+    zeo2_ini
+    zeo1_conf
+    zeo2_conf
+    zeo_client1
+    zeo_client2
+
+[zeo_server]
+recipe = zc.zodbrecipes:server
+zeo.conf =
+  <zeo>
+    address 8100
+  </zeo>
+  <blobstorage 1>
+    blob-dir ${zope_conf:blobstorage}
+    <filestorage 1>
+      path ${zope_conf:filestorage}/Data.fs
+    </filestorage>
+  </blobstorage>
+  <eventlog>
+    # This sets up logging to a file.
+    # The "path" setting can be a relative or absolute
+    # filesystem path.
+    <logfile>
+      path ${zope_conf:logfiles}/z3.log
+      formatter zope.exceptions.log.Formatter
+    </logfile>
+    # This sets up logging to to standard output.
+    # The "path" setting can be the tokens STDOUT or STDERR
+    #  <logfile>
+    #    path STDOUT
+    #    formatter zope.exceptions.log.Formatter
+    #  </logfile>
+  </eventlog>
+
+[zope_conf_zeo_1]
+recipe = collective.recipe.template
+input = etc/zope_zeo1.conf.in
+output = ${buildout:parts-directory}/etc/zope_zeo1.conf
+filestorage = ${buildout:directory}/var/filestorage
+blobstorage = ${buildout:directory}/var/blobstorage
+logfiles = ${buildout:directory}/var/log
+extra =
+# extra = <grokwarnings>True</grokwarnings>
+# 'extra' is copied verbatim. Use it for product config sections and so.
+
+[zope_conf_zeo_2]
+recipe = collective.recipe.template
+input = etc/zope_zeo2.conf.in
+output = ${buildout:parts-directory}/etc/zope_zeo2.conf
+filestorage = ${buildout:directory}/var/filestorage
+blobstorage = ${buildout:directory}/var/blobstorage
+logfiles = ${buildout:directory}/var/log
+extra =
+# extra = <grokwarnings>True</grokwarnings>
+# 'extra' is copied verbatim. Use it for product config sections and so.
+
+[zeo1_ini]
+recipe = collective.recipe.template
+input = etc/zeo1.ini.in
+output = ${buildout:parts-directory}/etc/zeo1.ini
+
+[zeo2_ini]
+recipe = collective.recipe.template
+input = etc/zeo2.ini.in
+output = ${buildout:parts-directory}/etc/zeo2.ini
+
+[zeo1_conf]
+recipe = collective.recipe.template
+input = etc/zeo1.conf.in
+output = ${buildout:parts-directory}/etc/zeo1.conf
+
+[zeo2_conf]
+recipe = collective.recipe.template
+input = etc/zeo2.conf.in
+output = ${buildout:parts-directory}/etc/zeo2.conf
+
+
+[zeo_client1]
+# See http://pypi.python.org/pypi/zc.recipe.egg for details...
+recipe = zc.recipe.egg
+eggs = waeup.custom [beaker]
+arguments = "${buildout:parts-directory}/etc/zeo1.conf"
+scripts = kofactl=zeo_client1
+
+[zeo_client2]
+# See http://pypi.python.org/pypi/zc.recipe.egg for details...
+recipe = zc.recipe.egg
+eggs = waeup.custom [beaker]
+arguments = "${buildout:parts-directory}/etc/zeo2.conf"
+scripts = kofactl=zeo_client2
+
Index: /main/waeup.custom/trunk/buildout.cfg
===================================================================
--- /main/waeup.custom/trunk/buildout.cfg	(revision 8035)
+++ /main/waeup.custom/trunk/buildout.cfg	(revision 8036)
@@ -13,4 +13,5 @@
     deploy_ini
     debug_ini
+    profile_ini
     waeupdocs
     coverage-detect
@@ -67,4 +68,6 @@
        PasteScript
        PasteDeploy
+       repoze.profile
+       ZODB3
 interpreter = python-console
 
@@ -150,4 +153,9 @@
 output = ${buildout:parts-directory}/etc/debug.ini
 
+[profile_ini]
+recipe = collective.recipe.template
+input = etc/profile.ini.in
+output = ${buildout:parts-directory}/etc/profile.ini
+
 [eggbasket]
 recipe = z3c.recipe.eggbasket
Index: /main/waeup.custom/trunk/etc/profile.ini.in
===================================================================
--- /main/waeup.custom/trunk/etc/profile.ini.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/profile.ini.in	(revision 8036)
@@ -0,0 +1,66 @@
+# profile.ini
+#
+# Profiling configuration for use with paster/WSGI
+#
+# Call http://localhost:8080/__profile__ to see the profiler in action
+
+[loggers]
+keys = root, wsgi
+
+[handlers]
+keys = console, accesslog
+
+[formatters]
+keys = generic, accesslog
+
+[logger_root]
+level = INFO
+handlers = console
+
+[logger_wsgi]
+level = INFO
+handlers = accesslog
+qualname = wsgi
+propagate = 0
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[handler_accesslog]
+class = FileHandler
+args = (os.path.join(r'${zope_conf:logfiles}', 'access.log'),
+        'a')
+level = INFO
+formatter = accesslog
+
+[formatter_generic]
+format = %(asctime)s %(levelname)s [%(name)s] %(message)s
+
+[formatter_accesslog]
+format = %(message)s
+
+[filter:translogger]
+use = egg:Paste#translogger
+setup_console_handler = False
+logger_name = wsgi
+
+[filter:profiler]
+use = egg:repoze.profile#profile
+
+[app:app]
+use = egg:waeup.custom
+
+[pipeline:main]
+pipeline = translogger profiler app
+
+[server:main]
+use = egg:Paste#http
+host = 0.0.0.0
+port = 8080
+
+[DEFAULT]
+# set the name of the zope.conf file
+zope_conf = %(here)s/zope.conf
Index: /main/waeup.custom/trunk/etc/zeo1.conf.in
===================================================================
--- /main/waeup.custom/trunk/etc/zeo1.conf.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zeo1.conf.in	(revision 8036)
@@ -0,0 +1,8 @@
+<runner>
+  program ${buildout:directory}/bin/paster serve --pid-file ${buildout:directory}/var/zeo1.pid ${buildout:directory}/parts/etc/zeo1.ini
+  daemon on
+  transcript ${zope_conf:logfiles}/zdaemon-zeo1.log
+  socket-name ${zope_conf:logfiles}/zeo1.sock
+  # Enable this to run the child process as a different user
+  # user zope
+</runner>
Index: /main/waeup.custom/trunk/etc/zeo1.ini.in
===================================================================
--- /main/waeup.custom/trunk/etc/zeo1.ini.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zeo1.ini.in	(revision 8036)
@@ -0,0 +1,62 @@
+# deploy.ini
+#
+# Deployment configuration for use with paster/WSGI
+#
+
+[loggers]
+keys = root, wsgi
+
+[handlers]
+keys = console, accesslog
+
+[formatters]
+keys = generic, accesslog
+
+[logger_root]
+level = INFO
+handlers = console
+
+[logger_wsgi]
+level = INFO
+handlers = accesslog
+qualname = wsgi
+propagate = 0
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[handler_accesslog]
+class = FileHandler
+args = (os.path.join(r'${zope_conf:logfiles}', 'zeo1.log'),
+        'a')
+level = INFO
+formatter = accesslog
+
+[formatter_generic]
+format = %(asctime)s %(levelname)s [%(name)s] %(message)s
+
+[formatter_accesslog]
+format = %(message)s
+
+[filter:translogger]
+use = egg:Paste#translogger
+setup_console_handler = False
+logger_name = wsgi
+
+[app:main]
+use = egg:waeup.custom
+filter-with = translogger
+
+[server:main]
+use = egg:Paste#http
+host = 0.0.0.0
+port = 8081
+threadpool_workers = 50
+use_threadpool = True
+
+[DEFAULT]
+# set the name of the zope.conf file
+zope_conf = %(here)s/zope_zeo1.conf
Index: /main/waeup.custom/trunk/etc/zeo2.conf.in
===================================================================
--- /main/waeup.custom/trunk/etc/zeo2.conf.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zeo2.conf.in	(revision 8036)
@@ -0,0 +1,8 @@
+<runner>
+  program ${buildout:directory}/bin/paster serve --pid-file ${buildout:directory}/var/zeo2.pid ${buildout:directory}/parts/etc/zeo2.ini
+  daemon on
+  transcript ${zope_conf:logfiles}/zdaemon-zeo2.log
+  socket-name ${zope_conf:logfiles}/zeo2.sock
+  # Enable this to run the child process as a different user
+  # user zope
+</runner>
Index: /main/waeup.custom/trunk/etc/zeo2.ini.in
===================================================================
--- /main/waeup.custom/trunk/etc/zeo2.ini.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zeo2.ini.in	(revision 8036)
@@ -0,0 +1,62 @@
+# deploy.ini
+#
+# Deployment configuration for use with paster/WSGI
+#
+
+[loggers]
+keys = root, wsgi
+
+[handlers]
+keys = console, accesslog
+
+[formatters]
+keys = generic, accesslog
+
+[logger_root]
+level = INFO
+handlers = console
+
+[logger_wsgi]
+level = INFO
+handlers = accesslog
+qualname = wsgi
+propagate = 0
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[handler_accesslog]
+class = FileHandler
+args = (os.path.join(r'${zope_conf:logfiles}', 'zeo2.log'),
+        'a')
+level = INFO
+formatter = accesslog
+
+[formatter_generic]
+format = %(asctime)s %(levelname)s [%(name)s] %(message)s
+
+[formatter_accesslog]
+format = %(message)s
+
+[filter:translogger]
+use = egg:Paste#translogger
+setup_console_handler = False
+logger_name = wsgi
+
+[app:main]
+use = egg:waeup.custom
+filter-with = translogger
+
+[server:main]
+use = egg:Paste#http
+host = 0.0.0.0
+port = 8082
+threadpool_workers = 50
+use_threadpool = True
+
+[DEFAULT]
+# set the name of the zope.conf file
+zope_conf = %(here)s/zope_zeo2.conf
Index: /main/waeup.custom/trunk/etc/zope_zeo1.conf.in
===================================================================
--- /main/waeup.custom/trunk/etc/zope_zeo1.conf.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zope_zeo1.conf.in	(revision 8036)
@@ -0,0 +1,88 @@
+# Identify the component configuration used to define the site:
+site-definition ${site_zcml:output}
+
+<zodb>
+
+  #cache-size
+
+  # num of concurrent pre-opened ZODB threads; default: 7
+  pool-size 25
+
+  # Standard blob storage
+#  <blobstorage>
+#    blob-dir ${zope_conf:blobstorage}
+#    <filestorage>
+#      path ${zope_conf:filestorage}/Data.fs
+#    </filestorage>
+#  </blobstorage>
+
+# Uncomment this if you want a blob-less standard file storage instead:
+#  <filestorage>
+#       path ${zope_conf:filestorage}
+#  </filestorage>
+
+# Uncomment this if you want to connect to a ZEO server instead:
+  <zeoclient>
+    server localhost:8100
+    storage 1
+    # ZEO client cache, in bytes
+    cache-size 20MB
+    # Uncomment to have a persistent disk cache
+    client zeo1
+  </zeoclient>
+
+# Uncomment this if you want to run a ZEO server instead:
+#  <zeo>
+#    # required; 'host:port' for TCP/IP, path to a socket file for UDP
+#    address localhost:8090
+#
+#    # optional; 'address' remarks apply
+#    #monitor-address localhost:8091
+#
+#    # optional; keep queue of last N transactions. Speeds up client cache verif.
+#    #invalidation-queue-size 10
+#
+#    # optional; is this a read-only instance? true or false
+#    #read-only false
+#
+#    # optional; seconds to wait for client to commit after acquiring lock
+#    #transaction-timeout 20
+#
+#    # optional; authentication protocol. With ZEO only 'digest' supported
+#    #authentication-protocol digest 
+#
+#    # optional; Path of the database containing authentication credentials.
+#    #authentication-database
+#
+#    # optional; the authentication realm of server
+#    #authentication-realm myrealm
+#  </zeo>
+
+</zodb>
+
+<eventlog>
+  # This sets up logging to a file.
+  # The "path" setting can be a relative or absolute
+  # filesystem path.
+
+  <logfile>
+    path ${zope_conf:logfiles}/zeo1_z3.log
+    formatter zope.exceptions.log.Formatter
+  </logfile>
+
+  # This sets up logging to to standard output.
+  # The "path" setting can be the tokens STDOUT or STDERR
+  
+#  <logfile>
+#    path STDOUT
+#    formatter zope.exceptions.log.Formatter
+#  </logfile>
+</eventlog>
+
+# Comment this line to disable developer mode.  This should be done in
+# production
+#devmode on
+
+# Extra configuration lines can be added to zope_conf's extra option. Put for 
+# instance productconf sections in here.
+${zope_conf:extra}
Index: /main/waeup.custom/trunk/etc/zope_zeo2.conf.in
===================================================================
--- /main/waeup.custom/trunk/etc/zope_zeo2.conf.in	(revision 8036)
+++ /main/waeup.custom/trunk/etc/zope_zeo2.conf.in	(revision 8036)
@@ -0,0 +1,88 @@
+# Identify the component configuration used to define the site:
+site-definition ${site_zcml:output}
+
+<zodb>
+
+  #cache-size
+
+  # num of concurrent pre-opened ZODB threads; default: 7
+  pool-size 7
+
+  # Standard blob storage
+#  <blobstorage>
+#    blob-dir ${zope_conf:blobstorage}
+#    <filestorage>
+#      path ${zope_conf:filestorage}/Data.fs
+#    </filestorage>
+#  </blobstorage>
+
+# Uncomment this if you want a blob-less standard file storage instead:
+#  <filestorage>
+#       path ${zope_conf:filestorage}
+#  </filestorage>
+
+# Uncomment this if you want to connect to a ZEO server instead:
+  <zeoclient>
+    server localhost:8100
+    storage 1
+    # ZEO client cache, in bytes
+    cache-size 20MB
+    # Uncomment to have a persistent disk cache
+    client zeo2
+  </zeoclient>
+
+# Uncomment this if you want to run a ZEO server instead:
+#  <zeo>
+#    # required; 'host:port' for TCP/IP, path to a socket file for UDP
+#    address localhost:8090
+#
+#    # optional; 'address' remarks apply
+#    #monitor-address localhost:8091
+#
+#    # optional; keep queue of last N transactions. Speeds up client cache verif.
+#    #invalidation-queue-size 10
+#
+#    # optional; is this a read-only instance? true or false
+#    #read-only false
+#
+#    # optional; seconds to wait for client to commit after acquiring lock
+#    #transaction-timeout 20
+#
+#    # optional; authentication protocol. With ZEO only 'digest' supported
+#    #authentication-protocol digest 
+#
+#    # optional; Path of the database containing authentication credentials.
+#    #authentication-database
+#
+#    # optional; the authentication realm of server
+#    #authentication-realm myrealm
+#  </zeo>
+
+</zodb>
+
+<eventlog>
+  # This sets up logging to a file.
+  # The "path" setting can be a relative or absolute
+  # filesystem path.
+
+  <logfile>
+    path ${zope_conf:logfiles}/zeo2_z3.log
+    formatter zope.exceptions.log.Formatter
+  </logfile>
+
+  # This sets up logging to to standard output.
+  # The "path" setting can be the tokens STDOUT or STDERR
+  
+#  <logfile>
+#    path STDOUT
+#    formatter zope.exceptions.log.Formatter
+#  </logfile>
+</eventlog>
+
+# Comment this line to disable developer mode.  This should be done in
+# production
+#devmode on
+
+# Extra configuration lines can be added to zope_conf's extra option. Put for 
+# instance productconf sections in here.
+${zope_conf:extra}
