Index: in/waeup.kofa/trunk/src/waeup/kofa/app.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/app.txt	(revision 12947)
+++ 	(revision )
@@ -1,146 +1,0 @@
-:mod:`waeup.kofa.app` -- central components for Kofa
-****************************************************
-
-.. :doctest:
-.. :layer: waeup.kofa.testing.KofaUnitTestLayer
-
-.. module:: waeup.kofa.app
-
-.. class:: University
-
-  The main Kofa application object is given with
-  :class:`University`. It provides the main containers as faculties,
-  hostels, etc.
-
-  .. attribute:: name
-
-     A string. The name of a university.
-
-  .. attribute:: faculties
-
-     An arbitrary object containing "faculties". In the case of
-     `University` it is a container of type
-     `waeup.kofa.interfaces.IFacultiesContainer`.
-
-
-Creating `University` instances
-===============================
-
-As :class:`University` instances make use of the Zope Component
-Architecture (ZCA), we have to setup the basic component registries,
-before we can create instances. We do this by simply grokking the
-whole :mod:`waeup` package. This way all interfaces, utilities and adapters
-defined in the package are looked up and registered with the global
-registries (this is done by the testlayer automatically).
-
-Now we can import the :class:`University` class and create an
-instance:
-
-    >>> from waeup.kofa.app import University
-    >>> myuniversity = University()
-    >>> myuniversity
-    <waeup.kofa.app.University object at 0x...>
-
-Instances of `University` comply with the interface
-`waeup.kofa.interfaces.IUniversity`:
-
-    >>> from zope.interface.verify import verifyClass
-    >>> from waeup.kofa.interfaces import IUniversity
-    >>> verifyClass(IUniversity, University)
-    True
-
-A freshly created instance provides the attributes promised by the
-interface:
-
-    >>> from waeup.kofa.app import University
-    >>> myuniversity = University()
-    >>> myuniversity['configuration'].name
-    u'Sample University'
-
-    >>> myuniversity['faculties']
-    <waeup.kofa.university.facultiescontainer.FacultiesContainer object at 0x...>
-
-  >>> myuniversity['students']
-  <waeup.kofa.students.container.StudentsContainer object at 0x...>
-
-  >>> myuniversity['users']
-  <waeup.kofa.userscontainer.UsersContainer object at 0x...>
-
-  >>> myuniversity['datacenter']
-  <waeup.kofa.datacenter.DataCenter object at 0x...>
-
-  >>> myuniversity['configuration']
-  <waeup.kofa.configuration.ConfigurationContainer object at 0x...>
-
-We can export universities. For this we lookup an appropriate exporter
-first::
-
-  >>> from waeup.kofa.interfaces import IKofaExporter
-  >>> exporter = IKofaExporter(myuniversity)
-  >>> exporter
-  <waeup.kofa.utils.importexport.Exporter object at 0x...>
-
-Now we can trigger the export::
-
-  >>> exporter.export()
-  <cStringIO.StringO object at 0x...>
-
-We can also get an XML representation as file. If we do not provide a
-filepath, we will get an instance of `cStringIO.StringIO`, i.e. a
-memory file::
-
-  >>> from waeup.kofa.interfaces import IKofaXMLExporter
-  >>> exporter = IKofaXMLExporter(myuniversity)
-  >>> f = exporter.export()
-  >>> f
-  <cStringIO.StringO object at 0x...>
-
-  >>> print f.read()
-  <?xml version="1.0" encoding="utf-8" ?>
-  <pickle>
-    <initialized_object id="...">
-  ...
-  </pickle>
-
-Kofa plugins
-============
-
-waeup.kofa provides an API to 'plugin' components. Things that should
-be setup at creation time of a Kofa application can indicate
-that by providing a utility providing IKofaPlugin.
-
-The plugins are looked up by an created app, which then will call the
-``setup()`` method of each plugin.
-
-   >>> from waeup.kofa.interfaces import IKofaPluggable
-   >>> from zope.component import getAdapters, getUtilitiesFor
-   >>> sorted(list(getUtilitiesFor(IKofaPluggable)))
-   [(u'accesscodes', <waeup.kofa.accesscodes...AccessCodePlugin ...)]
-
-
-We can provide a new plugin like this:
-
-   >>> import grok
-   >>> from waeup.kofa.interfaces import IKofaPluggable
-   >>> class MyPlugin(grok.GlobalUtility):
-   ...   grok.implements(IKofaPluggable)
-   ...   def setup(self, site, name, logger):
-   ...     print "Setup was called for"
-   ...     print site
-   ...   def update(self, site, name, logger):
-   ...     pass
-
-When we register the plugin
-
-   >>> grok.testing.grok_component('MyPlugin', MyPlugin)
-   True
-
-and setup a new Kofa instance, we will get a message:
-
-   >>> from waeup.kofa.app import University
-   >>> site = University()
-   Setup was called for
-   <waeup.kofa.app.University object at 0x...>
-
-Apparently the plugin can do with the University object whatever it
-likes. That's what plugins are for.
Index: in/waeup.kofa/trunk/src/waeup/kofa/authentication.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/authentication.txt	(revision 12947)
+++ 	(revision )
@@ -1,212 +1,0 @@
-Kofa authentication
-*******************
-
-We need to protect most pieces of our portals from unauthenticated
-access.
-
-Therefore users have to login to access main functionality and they
-are able to log out afterwards.
-
-Before we can check access we have to create an app:
-
-  >>> from zope.component.hooks import setSite # only needed in tests
-  >>> from waeup.kofa.app import University
-  >>> root = getRootFolder()
-  >>> u = University()
-  >>> root['app'] = u
-  >>> setSite(root['app'])                     # only needed in tests
-
-To make sure, we can 'watch' pages, we first have to initialize our
-test browser:
-
-  >>> from zope.testbrowser.testing import Browser
-  >>> browser = Browser()
-  >>> browser.handleErrors = False
-
-Creating users (principals)
-===========================
-
-Before we can login, we have to provide a user (``principal`` in Zope
-terms) with a password (and optional a title or description):
-
-  >>> root['app']['users'].addUser('bob', 'bobsecret',
-  ...                           title='Bob', description='A sample user')
-
-We can also add complete `Account` objects. An `Account` stores the
-user credentials and some metadata persistently:
-
-  >>> from waeup.kofa.authentication import Account
-  >>> alice = Account('alice', 'alicesecret',roles=['waeup.ManageDataCenter'])
-  >>> root['app']['users'].addAccount(alice)
-
-See ``userscontainer.txt`` for details about the UsersContainer we use here.
-
-Users and local roles
-=====================
-
-Accounts also hold infos about local roles assigned to a user. In the
-beginning, users have the local owner role of their own account object:
-
-  >>> alice.getLocalRoles()
-  {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
-
-User  automatically get the global AcademicsOfficer role:
-
-  >>> alice.getSiteRolesForPrincipal()
-  ['waeup.ManageDataCenter', 'waeup.AcademicsOfficer']
-
-We can tell an account, that Alice got some role for another object:
-
-  >>> chalet = object()
-  >>> root['app']['chalet'] = chalet
-  >>> alice.notifyLocalRoleChanged(chalet, 'BigBoss', granted=True)
-
-Now Alice is the Big Boss:
-
-  >>> alice.getLocalRoles()
-  {'BigBoss': [<object object at 0x...>]}
-
-When we do not want Alice to be the Big Boss we can tell that too:
-
-  >>> alice.notifyLocalRoleChanged(chalet, 'BigBoss', granted=False)
-  >>> alice.getLocalRoles()
-  {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
-
-We can also use events to trigger such actions. This is recommended
-because we do not neccessarily know where Alice lives:
-
-  >>> from waeup.kofa.authentication import LocalRoleSetEvent
-  >>> from zope.event import notify
-  >>> notify(LocalRoleSetEvent(chalet, 'BigBoss', 'alice',
-  ...                          granted=True))
-  >>> alice.getLocalRoles()
-  {'BigBoss': [<object object at 0x...>]}
-
-When objects are deleted, local roles are also deleted
-semi-magically. This happens through event subscribers listening to
-IObjectRemovedEvents. The latters are naturally only fired when ZODB
-stored objects are removed. Furthermore this subscriber reads the
-internal local roles table.
-
-We create a faculty and grant Bob a local role:
-
-   >>> from zope.securitypolicy.interfaces import IPrincipalRoleManager
-   >>> from waeup.kofa.university.faculty import Faculty
-   >>> faculty = Faculty()
-   >>> root['app']['bobs_fac'] = faculty
-   >>> role_manager = IPrincipalRoleManager(faculty)
-   >>> role_manager.assignRoleToPrincipal(
-   ...    'waeup.PortalManager', 'bob')
-
-We notify the machinery about that fact:
-
-   >>> notify(LocalRoleSetEvent(faculty, 'waeup.PortalManager', 'bob',
-   ...                          granted=True))
-   >>> bob = root['app']['users']['bob']
-   >>> bob.getLocalRoles()
-   {'waeup.PortalManager': [<waeup.kofa...Faculty object at 0x...>]}
-
-When we delete the faculty from ZODB, also Bobs roles are modified:
-
-   >>> del root['app']['bobs_fac']
-   >>> bob.getLocalRoles()
-   {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
-
-If one notifies the machinery of a local role removal for an object
-that cannot have local roles, this will cause no trouble:
-
-   >>> mycontext = None #object()
-   >>> notify(LocalRoleSetEvent(
-   ...   mycontext, 'waeup.PortalManager', 'bob', granted=False
-   ... )) is None
-   True
-
-When an account get deleted, also the local roles of the owner get
-removed. Let's setup a local role for `alice`:
-
-   >>> faculty = Faculty()
-   >>> root['app']['alice_fac'] = faculty
-   >>> role_manager = IPrincipalRoleManager(faculty)
-   >>> role_manager.assignRoleToPrincipal(
-   ...    'waeup.PortalManager', 'alice')
-   >>> notify(LocalRoleSetEvent(faculty, 'waeup.PortalManager', 'alice',
-   ...                          granted=True))
-
-The local role is set now:
-
-   >>> from zope.securitypolicy.interfaces import IPrincipalRoleMap
-   >>> IPrincipalRoleMap(faculty).getPrincipalsAndRoles()
-   [('waeup.PortalManager', 'alice', PermissionSetting: Allow)]
-
-But when we delete Alices account from ZODB:
-
-   >>> del root['app']['users']['alice']
-   >>> IPrincipalRoleMap(faculty).getPrincipalsAndRoles()
-   []
-
-the local role has gone and also the site roles have been removed:
-
-   >>> prm = IPrincipalRoleManager(root['app'])
-   >>> [x[0] for x in prm.getRolesForPrincipal('alice')
-   ...      if x[0].startswith('waeup.')]
-   []
-
-
-Logging in via side bar
-=======================
-
-We can access the front page without restrictions:
-
-  >>> browser.open('http://localhost/app')
-  >>> print browser.headers['Status']
-  200 Ok
-
-We have to go to one of the login pages first:
-
-  >>> browser.open('http://localhost/app')
-  >>> browser.getLink('Login').click()
-  >>> print browser.headers['Status']
-  200 Ok
-
-There is a login form on tis page:
-
-  >>> 'form.login' in browser.contents
-  True
-
-  >>> 'form.logout' in browser.contents
-  False
-
-We use this form:
-
-  >>> browser.getControl(name='form.login').value = 'bob'
-  >>> browser.getControl(name='form.password').value = 'invalidpw'
-  >>> browser.getControl('Login').click()
-  >>> 'You entered invalid credentials' in browser.contents
-  True
-
-  >>> browser.getControl(name='form.login').value = 'bob'
-  >>> browser.getControl(name='form.password').value = 'bobsecret'
-  >>> browser.getControl('Login').click()
-
-Now the login form is gone. Instead we have the opportunity to logout:
-
-  >>> 'form.login' in browser.contents
-  False
-
-  >>> logout = browser.getLink('Logout')
-  >>> logout
-  <Link text='Logout' url='http://localhost/app/@@logout'>
-
-The user title is also displayed in the sidebar:
-
-  >>> 'Bob' in browser.contents
-  True
-
-We can also log out afterwards:
-
-  >>> logout.click()
-  >>> print browser.contents
-  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"...
-  ...Login
-  ...
-
Index: in/waeup.kofa/trunk/src/waeup/kofa/catalog.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/catalog.txt	(revision 12947)
+++ 	(revision )
@@ -1,230 +1,0 @@
-Cataloging Support
-******************
-
-.. module:: waeup.kofa.catalog
-
-Components that support cataloging and searching objects inside a
-Kofa site.
-
-.. :doctest:
-.. :layer: waeup.kofa.testing.KofaUnitTestLayer
-
-The KofaQuery Class
-===================
-
-.. class:: KofaQuery()
-
-   .. attribute:: grok.implements(hurry.query.interfaces.IQuery)
-
-   A `hurry.query.query.Query` compatible query that also supports
-   retrival of plain ``Bree`` result sets as used inside a catalog.
-
-   Like `hurry.query.query.Query` objects, `KofaQuery` is some kind
-   of a meta query or 'compound query' that can give the cataloged
-   objects (or their int ids) matching one or more 'subqueries'.
-
-   This way you can search for objects (or their int ids) that match
-   several criteria at the same time. See ``examples`` section below.
-
-   A singleton instance of this class is also available as global
-   utility.
-
-   .. method:: searchResults(query)
-
-     Get the cataloged objects determined by ``query``.
-
-   .. method:: apply(query)
-
-     Get the list of int ids (a `BTree` result set) for objects
-     determined by ``query``.
-
-     The list of int ids is less expensive to compute than the
-     complete search results and sufficient, for instance, when you
-     only need the number of objects that match a query and not the
-     objects themselves.
-
-Examples
-========
-
-Getting a general query object
-------------------------------
-
-We can get a KofaQuery object by asking for an unnamed global utility
-implementing `hurry.query.interfaces.IQuery`:
-
-    >>> from hurry.query.interfaces import IQuery
-    >>> from zope.component import getUtility
-    >>> q = getUtility(IQuery)
-    >>> q
-    <waeup.kofa.catalog.KofaQuery object at 0x...>
-
-This query can get 'subqueries' and delivers the objects found or
-their ids. To show this we have to setup a catalog with some entries.
-
-
-Setting up a catalog and feeding it
------------------------------------
-
-    >>> from zope.catalog.interfaces import ICatalog
-    >>> from zope.catalog.catalog import Catalog
-    >>> mycat = Catalog()
-
-We register this catalog with the component architechture as a utility
-named 'mycatalog':
-
-    >>> from zope.component import provideUtility
-    >>> provideUtility(mycat, ICatalog, 'mycatalog')
-
-We setup a special content type whose instances we will catalog later:
-
-    >>> from zope.interface import Interface, Attribute, implements
-    >>> from zope.container.contained import Contained
-    >>> class IMammoth(Interface):
-    ...   name = Attribute('name')
-    ...   age = Attribute('age')
-
-    >>> class Mammoth(Contained):
-    ...   implements(IMammoth)
-    ...   def __init__(self, name, age):
-    ...     self.name = name
-    ...     self.age = age
-    ...   def __cmp__(self, other):
-    ...     return cmp(self.name, other.name)
-
-By including the __cmp__ method we make sure search results can be
-stably sorted.
-
-We also setup a `zope.intid.interfaces.IIntIds` utility. This is not
-necessary for plain catalogs, but when we want to use KofaQuery (or
-`hurry.query.query.Query` objects), as to get a unique mapping from
-objects (stored in ZODB) to integer numbers (stored in catalogs),
-these query objects lookup a global IIntIds utiliy:
-
-    >>> from zope import interface
-    >>> import zope.intid.interfaces
-    >>> class DummyIntId(object):
-    ...     interface.implements(zope.intid.interfaces.IIntIds)
-    ...     MARKER = '__dummy_int_id__'
-    ...     def __init__(self):
-    ...         self.counter = 0
-    ...         self.data = {}
-    ...     def register(self, obj):
-    ...         intid = getattr(obj, self.MARKER, None)
-    ...         if intid is None:
-    ...             setattr(obj, self.MARKER, self.counter)
-    ...             self.data[self.counter] = obj
-    ...             intid = self.counter
-    ...             self.counter += 1
-    ...         return intid
-    ...     def getObject(self, intid):
-    ...         return self.data[intid]
-    ...     def __iter__(self):
-    ...         return iter(self.data)
-    >>> intid = DummyIntId()
-    >>> from zope.component import provideUtility
-    >>> provideUtility(intid, zope.intid.interfaces.IIntIds)
-
-Now we can catalog some mammoths. Here we create a herd and catalog
-each item of it:
-
-    >>> from zope.catalog.field import FieldIndex
-    >>> mycat['mammoth_name'] = FieldIndex('name', IMammoth)
-    >>> mycat['mammoth_age'] = FieldIndex('age', IMammoth)
-
-    >>> herd = [
-    ...   Mammoth(name='Fred', age=33),
-    ...   Mammoth(name='Hank', age=30),
-    ...   Mammoth(name='Wilma', age=28),
-    ... ]
-
-    >>> for mammoth in herd:
-    ...   mycat.index_doc(intid.register(mammoth), mammoth)
-
-
-Searching for result sets
--------------------------
-
-Finally we can perform queries:
-
-    >>> from hurry.query import Eq
-    >>> from zope.component import getUtility
-    >>> subquery1 = Eq(('mycatalog', 'mammoth_name'), 'Fred')
-
-The latter means: search for objects whose name is ``'Fred'`` in the
-``mammoth_name`` index of a catalog registered as a utility named
-``mycatalog``.
-
-    >>> from hurry.query import Between
-    >>> subquery2 = Between(('mycatalog', 'mammoth_age'), 30, 33)
-
-This means: ask for objects cataloged in an index named 'mammoth_age',
-whose cataloged value is between 30 and 33 (including this values).
-
-    >>> r1 = q.apply(subquery2)
-    >>> r1
-    IFSet([0, 1])
-
-Using ``apply()`` above, we get a set of values stored in an
-``IFBTree``:
-
-    >>> type(r1)
-    <type 'BTrees.IFBTree.IFSet'>
-
-``IFBTree`` objects implement a rather efficient integer to float
-mapping where also integers are allowed as values. For each object
-found (i.e. mammoths whose age is between 30 and 33), we get the
-number of its entry.
-
-To get the real object, we can use intids here, because we setup an
-appropriate IIntIds utility before:
-
-    >>> [intid.getObject(x).name for x in r1]
-    ['Fred', 'Hank']
-
-We can (and should) also use the `searchResults()` method explained
-below to do that.
-
-Retrieving BTree sets can, however, make sense, if you want to know
-only the number of results for a particular query or whether there are
-results at all in a more efficient way:
-
-    >>> len(r1)
-    2
-
-Searching for objects
----------------------
-
-Very often we don't want to know the catalog-internal 'ids' of
-searched objects but the objects themselves.
-
-This can be done by using the ``searchResults`` method of
-``KofaQuery``:
-
-    >>> r2 = q.searchResults(subquery1)
-    >>> r2
-    <zope.catalog.catalog.ResultSet instance at 0x...>
-
-    >>> list(r2)
-    [<Mammoth object at 0x...>]
-
-We got one result item, we can immediately ask for further infos. To
-access a result item by its index number, we have to turn the
-ResultSet into an ordinary list before:
-
-    >>> entry = list(r2)[0]
-    >>> entry.name, entry.age
-    ('Fred', 33)
-
-We can also use ``subquery2`` as above:
-
-    >>> r3 = q.searchResults(subquery2)
-    >>> [(x.name, x.age) for x in r3]
-    [('Fred', 33), ('Hank', 30)]
-
-or use both queries at once:
-
-    >>> r4 = q.searchResults(subquery1 & subquery2)
-    >>> [(x.name, x.age) for x in r4]
-    [('Fred', 33)]
-
-which will give us, of course, the same result set as with subquery1.
Index: in/waeup.kofa/trunk/src/waeup/kofa/datacenter.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/datacenter.txt	(revision 12947)
+++ 	(revision )
@@ -1,290 +1,0 @@
-Data Center
-***********
-
-The Kofa data center cares for managing CSV files and importing then.
-
-.. :doctest:
-.. :layer: waeup.kofa.testing.KofaUnitTestLayer
-
-Creating a data center
-======================
-
-A data center can be created easily:
-
-    >>> from waeup.kofa.datacenter import DataCenter
-    >>> mydatacenter = DataCenter()
-    >>> mydatacenter
-    <waeup.kofa.datacenter.DataCenter object at 0x...>
-
-Each data center has a location in file system where files are stored:
-
-    >>> storagepath = mydatacenter.storage
-    >>> storagepath
-    '/tmp/tmp...'
-
-Beside other things it provides a location to put data of deleted
-items into:
-
-    >>> import os
-    >>> del_path = mydatacenter.deleted_path
-    >>> os.path.isdir(del_path)
-    True
-
-Overall it complies with the `IDataCenter` interface:
-
-    >>> from zope.interface import verify
-    >>> from waeup.kofa.interfaces import IDataCenter
-    >>> verify.verifyObject(IDataCenter, DataCenter() )
-    True
-
-    >>> verify.verifyClass(IDataCenter, DataCenter)
-    True
-
-Managing the storage path
-=========================
-
-We can set another storage path:
-
-    >>> import os
-    >>> os.mkdir('newlocation')
-    >>> newpath = os.path.abspath('newlocation')
-    >>> mydatacenter.setStoragePath(newpath)
-    []
-
-The result here is a list of filenames, that could not be
-copied. Luckily, this list is empty.
-
-When we set a new storage path, we can tell to move all files in the
-old location to the new one. To see this feature in action, we first
-have to put a file into the old location:
-
-    >>> open(os.path.join(newpath, 'myfile.txt'), 'wb').write('hello')
-
-Now we can set a new location and the file will be copied:
-
-    >>> verynewpath = os.path.abspath('verynewlocation')
-    >>> os.mkdir(verynewpath)
-
-    >>> mydatacenter.setStoragePath(verynewpath, move=True)
-    []
-
-    >>> storagepath = mydatacenter.storage
-    >>> 'myfile.txt' in os.listdir(verynewpath)
-    True
-
-We remove the created file to have a clean testing environment for
-upcoming examples:
-
-    >>> os.unlink(os.path.join(storagepath, 'myfile.txt'))
-
-Uploading files
-===============
-
-We can get a list of files stored in that location:
-
-    >>> mydatacenter.getPendingFiles()
-    []
-
-Let's put some file in the storage:
-
-    >>> import os
-    >>> filepath = os.path.join(storagepath, 'data.csv')
-    >>> open(filepath, 'wb').write('Some Content\n')
-
-Now we can find a file:
-
-    >>> mydatacenter.getPendingFiles()
-    [<waeup.kofa.datacenter.DataCenterFile object at 0x...>]
-
-As we can see, the actual file is wrapped by a convenience wrapper,
-that enables us to fetch some data about the file. The data returned
-is formatted in strings, so that it can easily be put into output
-pages:
-
-    >>> datafile = mydatacenter.getPendingFiles()[0]
-    >>> datafile.getSize()
-    '13 bytes'
-
-    >>> datafile.getDate() # Nearly current datetime...
-    '...'
-
-Clean up:
-
-    >>> import shutil
-    >>> shutil.rmtree(newpath)
-    >>> shutil.rmtree(verynewpath)
-
-
-Distributing processed files
-============================
-
-When files were processed by a batch processor, we can put the
-resulting files into desired destinations.
-
-We recreate the datacenter root in case it is missing:
-
-    >>> import os
-    >>> dc_root = mydatacenter.storage
-    >>> fin_dir = os.path.join(dc_root, 'finished')
-    >>> unfin_dir = os.path.join(dc_root, 'unfinished')
-
-    >>> def recreate_dc_storage():
-    ...   if os.path.exists(dc_root):
-    ...     shutil.rmtree(dc_root)
-    ...   os.mkdir(dc_root)
-    ...   mydatacenter.setStoragePath(mydatacenter.storage)
-    >>> recreate_dc_storage()
-
-We define a function that creates a set of faked result files:
-
-    >>> import os
-    >>> import tempfile
-    >>> def create_fake_results(source_basename, create_pending=True):
-    ...   tmp_dir = tempfile.mkdtemp()
-    ...   src = os.path.join(dc_root, source_basename)
-    ...   pending_src = None
-    ...   if create_pending:
-    ...     pending_src = os.path.join(tmp_dir, 'mypendingsource.csv')
-    ...   finished_src = os.path.join(tmp_dir, 'myfinishedsource.csv')
-    ...   for path in (src, pending_src, finished_src):
-    ...     if path is not None:
-    ...       open(path, 'wb').write('blah')
-    ...   return tmp_dir, src, finished_src, pending_src
-
-Now we can create the set of result files, that typically come after a
-successful processing of a regular source:
-
-Now we can try to distribute those files. Let's start with a source
-file, that was processed successfully:
-
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.csv', create_pending=False)
-    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
-    ...                            pending_src, mode='create')
-    >>> sorted(os.listdir(dc_root))
-    ['deleted', 'finished', 'logs', 'unfinished']
-
-    >>> sorted(os.listdir(fin_dir))
-    ['mysource.create.finished.csv', 'mysource.csv']
-
-    >>> sorted(os.listdir(unfin_dir))
-    []
-
-The created dir will be removed for us by the datacenter. This way we
-can assured, that less temporary dirs are left hanging around:
-
-    >>> os.path.exists(tmp_dir)
-    False
-
-The root dir is empty, while the original file and the file containing
-all processed data were moved to'finished/'.
-
-Now we restart, but this time we fake an erranous action:
-
-    >>> recreate_dc_storage()
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.csv')
-    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
-    ...                                 pending_src, mode='create')
-    >>> sorted(os.listdir(dc_root))
-    ['deleted', 'finished', 'logs', 'mysource.create.pending.csv', 'unfinished']
-
-    >>> sorted(os.listdir(fin_dir))
-    ['mysource.create.finished.csv']
-
-    >>> sorted(os.listdir(unfin_dir))
-    ['mysource.csv']
-
-While the original source was moved to the 'unfinished' dir, the
-pending file went to the root and the set of already processed items
-are stored in finished/.
-
-We fake processing the pending file and assume that everything went
-well this time:
-
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.create.pending.csv', create_pending=False)
-    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
-    ...                                 pending_src, mode='create')
-
-    >>> sorted(os.listdir(dc_root))
-    ['deleted', 'finished', 'logs', 'unfinished']
-
-    >>> sorted(os.listdir(fin_dir))
-    ['mysource.create.finished.csv', 'mysource.csv']
-
-    >>> sorted(os.listdir(unfin_dir))
-    []
-
-The result is the same as in the first case shown above.
-
-We restart again, but this time we fake several non-working imports in
-a row.
-
-We start with a faulty start-import:
-
-    >>> recreate_dc_storage()
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.csv')
-    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
-    ...                                 pending_src, mode='create')
-
-We try to process the pending file, which fails again:
-
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.create.pending.csv')
-    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
-    ...                                 pending_src, mode='create')
-
-We try to process the new pending file:
-
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.create.pending.csv')
-    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
-    ...                                 pending_src, mode='create')
-
-    >>> sorted(os.listdir(dc_root))
-    ['deleted', 'finished', 'logs', 'mysource.create.pending.csv', 'unfinished']
-
-    >>> sorted(os.listdir(fin_dir))
-    ['mysource.create.finished.csv']
-
-    >>> sorted(os.listdir(unfin_dir))
-    ['mysource.csv']
-
-Finally, we process the pending file and everything works:
-
-    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
-    ...  'mysource.create.pending.csv', create_pending=False)
-    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
-    ...                                 pending_src, mode='create')
-
-    >>> sorted(os.listdir(dc_root))
-    ['deleted', 'finished', 'logs', 'unfinished']
-
-    >>> sorted(os.listdir(fin_dir))
-    ['mysource.create.finished.csv', 'mysource.csv']
-
-    >>> sorted(os.listdir(unfin_dir))
-    []
-
-The root dir is empty (contains no input files) and only the files in
-finished-subdirectory remain.
-
-
-We can get a list of imported files stored in the finished subfolder:
-
-    >>> mydatacenter.getFinishedFiles()
-    [<waeup.kofa.datacenter.DataCenterFile object at ...>]
-
-    >>> datafile = mydatacenter.getFinishedFiles()[0]
-    >>> datafile.getSize()
-    '2 bytes'
-
-    >>> datafile.getDate() # Nearly current datetime...
-    '...'
-
-
-Clean up:
-
-    >>> shutil.rmtree(verynewpath)
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/app.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/app.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/app.txt	(revision 12948)
@@ -0,0 +1,146 @@
+:mod:`waeup.kofa.app` -- central components for Kofa
+****************************************************
+
+.. :doctest:
+.. :layer: waeup.kofa.testing.KofaUnitTestLayer
+
+.. module:: waeup.kofa.app
+
+.. class:: University
+
+  The main Kofa application object is given with
+  :class:`University`. It provides the main containers as faculties,
+  hostels, etc.
+
+  .. attribute:: name
+
+     A string. The name of a university.
+
+  .. attribute:: faculties
+
+     An arbitrary object containing "faculties". In the case of
+     `University` it is a container of type
+     `waeup.kofa.interfaces.IFacultiesContainer`.
+
+
+Creating `University` instances
+===============================
+
+As :class:`University` instances make use of the Zope Component
+Architecture (ZCA), we have to setup the basic component registries,
+before we can create instances. We do this by simply grokking the
+whole :mod:`waeup` package. This way all interfaces, utilities and adapters
+defined in the package are looked up and registered with the global
+registries (this is done by the testlayer automatically).
+
+Now we can import the :class:`University` class and create an
+instance:
+
+    >>> from waeup.kofa.app import University
+    >>> myuniversity = University()
+    >>> myuniversity
+    <waeup.kofa.app.University object at 0x...>
+
+Instances of `University` comply with the interface
+`waeup.kofa.interfaces.IUniversity`:
+
+    >>> from zope.interface.verify import verifyClass
+    >>> from waeup.kofa.interfaces import IUniversity
+    >>> verifyClass(IUniversity, University)
+    True
+
+A freshly created instance provides the attributes promised by the
+interface:
+
+    >>> from waeup.kofa.app import University
+    >>> myuniversity = University()
+    >>> myuniversity['configuration'].name
+    u'Sample University'
+
+    >>> myuniversity['faculties']
+    <waeup.kofa.university.facultiescontainer.FacultiesContainer object at 0x...>
+
+  >>> myuniversity['students']
+  <waeup.kofa.students.container.StudentsContainer object at 0x...>
+
+  >>> myuniversity['users']
+  <waeup.kofa.userscontainer.UsersContainer object at 0x...>
+
+  >>> myuniversity['datacenter']
+  <waeup.kofa.datacenter.DataCenter object at 0x...>
+
+  >>> myuniversity['configuration']
+  <waeup.kofa.configuration.ConfigurationContainer object at 0x...>
+
+We can export universities. For this we lookup an appropriate exporter
+first::
+
+  >>> from waeup.kofa.interfaces import IKofaExporter
+  >>> exporter = IKofaExporter(myuniversity)
+  >>> exporter
+  <waeup.kofa.utils.importexport.Exporter object at 0x...>
+
+Now we can trigger the export::
+
+  >>> exporter.export()
+  <cStringIO.StringO object at 0x...>
+
+We can also get an XML representation as file. If we do not provide a
+filepath, we will get an instance of `cStringIO.StringIO`, i.e. a
+memory file::
+
+  >>> from waeup.kofa.interfaces import IKofaXMLExporter
+  >>> exporter = IKofaXMLExporter(myuniversity)
+  >>> f = exporter.export()
+  >>> f
+  <cStringIO.StringO object at 0x...>
+
+  >>> print f.read()
+  <?xml version="1.0" encoding="utf-8" ?>
+  <pickle>
+    <initialized_object id="...">
+  ...
+  </pickle>
+
+Kofa plugins
+============
+
+waeup.kofa provides an API to 'plugin' components. Things that should
+be setup at creation time of a Kofa application can indicate
+that by providing a utility providing IKofaPlugin.
+
+The plugins are looked up by an created app, which then will call the
+``setup()`` method of each plugin.
+
+   >>> from waeup.kofa.interfaces import IKofaPluggable
+   >>> from zope.component import getAdapters, getUtilitiesFor
+   >>> sorted(list(getUtilitiesFor(IKofaPluggable)))
+   [(u'accesscodes', <waeup.kofa.accesscodes...AccessCodePlugin ...)]
+
+
+We can provide a new plugin like this:
+
+   >>> import grok
+   >>> from waeup.kofa.interfaces import IKofaPluggable
+   >>> class MyPlugin(grok.GlobalUtility):
+   ...   grok.implements(IKofaPluggable)
+   ...   def setup(self, site, name, logger):
+   ...     print "Setup was called for"
+   ...     print site
+   ...   def update(self, site, name, logger):
+   ...     pass
+
+When we register the plugin
+
+   >>> grok.testing.grok_component('MyPlugin', MyPlugin)
+   True
+
+and setup a new Kofa instance, we will get a message:
+
+   >>> from waeup.kofa.app import University
+   >>> site = University()
+   Setup was called for
+   <waeup.kofa.app.University object at 0x...>
+
+Apparently the plugin can do with the University object whatever it
+likes. That's what plugins are for.
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/authentication.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/authentication.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/authentication.txt	(revision 12948)
@@ -0,0 +1,212 @@
+Kofa authentication
+*******************
+
+We need to protect most pieces of our portals from unauthenticated
+access.
+
+Therefore users have to login to access main functionality and they
+are able to log out afterwards.
+
+Before we can check access we have to create an app:
+
+  >>> from zope.component.hooks import setSite # only needed in tests
+  >>> from waeup.kofa.app import University
+  >>> root = getRootFolder()
+  >>> u = University()
+  >>> root['app'] = u
+  >>> setSite(root['app'])                     # only needed in tests
+
+To make sure, we can 'watch' pages, we first have to initialize our
+test browser:
+
+  >>> from zope.testbrowser.testing import Browser
+  >>> browser = Browser()
+  >>> browser.handleErrors = False
+
+Creating users (principals)
+===========================
+
+Before we can login, we have to provide a user (``principal`` in Zope
+terms) with a password (and optional a title or description):
+
+  >>> root['app']['users'].addUser('bob', 'bobsecret',
+  ...                           title='Bob', description='A sample user')
+
+We can also add complete `Account` objects. An `Account` stores the
+user credentials and some metadata persistently:
+
+  >>> from waeup.kofa.authentication import Account
+  >>> alice = Account('alice', 'alicesecret',roles=['waeup.ManageDataCenter'])
+  >>> root['app']['users'].addAccount(alice)
+
+See ``userscontainer.txt`` for details about the UsersContainer we use here.
+
+Users and local roles
+=====================
+
+Accounts also hold infos about local roles assigned to a user. In the
+beginning, users have the local owner role of their own account object:
+
+  >>> alice.getLocalRoles()
+  {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
+
+User  automatically get the global AcademicsOfficer role:
+
+  >>> alice.getSiteRolesForPrincipal()
+  ['waeup.ManageDataCenter', 'waeup.AcademicsOfficer']
+
+We can tell an account, that Alice got some role for another object:
+
+  >>> chalet = object()
+  >>> root['app']['chalet'] = chalet
+  >>> alice.notifyLocalRoleChanged(chalet, 'BigBoss', granted=True)
+
+Now Alice is the Big Boss:
+
+  >>> alice.getLocalRoles()
+  {'BigBoss': [<object object at 0x...>]}
+
+When we do not want Alice to be the Big Boss we can tell that too:
+
+  >>> alice.notifyLocalRoleChanged(chalet, 'BigBoss', granted=False)
+  >>> alice.getLocalRoles()
+  {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
+
+We can also use events to trigger such actions. This is recommended
+because we do not neccessarily know where Alice lives:
+
+  >>> from waeup.kofa.authentication import LocalRoleSetEvent
+  >>> from zope.event import notify
+  >>> notify(LocalRoleSetEvent(chalet, 'BigBoss', 'alice',
+  ...                          granted=True))
+  >>> alice.getLocalRoles()
+  {'BigBoss': [<object object at 0x...>]}
+
+When objects are deleted, local roles are also deleted
+semi-magically. This happens through event subscribers listening to
+IObjectRemovedEvents. The latters are naturally only fired when ZODB
+stored objects are removed. Furthermore this subscriber reads the
+internal local roles table.
+
+We create a faculty and grant Bob a local role:
+
+   >>> from zope.securitypolicy.interfaces import IPrincipalRoleManager
+   >>> from waeup.kofa.university.faculty import Faculty
+   >>> faculty = Faculty()
+   >>> root['app']['bobs_fac'] = faculty
+   >>> role_manager = IPrincipalRoleManager(faculty)
+   >>> role_manager.assignRoleToPrincipal(
+   ...    'waeup.PortalManager', 'bob')
+
+We notify the machinery about that fact:
+
+   >>> notify(LocalRoleSetEvent(faculty, 'waeup.PortalManager', 'bob',
+   ...                          granted=True))
+   >>> bob = root['app']['users']['bob']
+   >>> bob.getLocalRoles()
+   {'waeup.PortalManager': [<waeup.kofa...Faculty object at 0x...>]}
+
+When we delete the faculty from ZODB, also Bobs roles are modified:
+
+   >>> del root['app']['bobs_fac']
+   >>> bob.getLocalRoles()
+   {'waeup.local.Owner': [<waeup.kofa.authentication.Account object at 0x...>]}
+
+If one notifies the machinery of a local role removal for an object
+that cannot have local roles, this will cause no trouble:
+
+   >>> mycontext = None #object()
+   >>> notify(LocalRoleSetEvent(
+   ...   mycontext, 'waeup.PortalManager', 'bob', granted=False
+   ... )) is None
+   True
+
+When an account get deleted, also the local roles of the owner get
+removed. Let's setup a local role for `alice`:
+
+   >>> faculty = Faculty()
+   >>> root['app']['alice_fac'] = faculty
+   >>> role_manager = IPrincipalRoleManager(faculty)
+   >>> role_manager.assignRoleToPrincipal(
+   ...    'waeup.PortalManager', 'alice')
+   >>> notify(LocalRoleSetEvent(faculty, 'waeup.PortalManager', 'alice',
+   ...                          granted=True))
+
+The local role is set now:
+
+   >>> from zope.securitypolicy.interfaces import IPrincipalRoleMap
+   >>> IPrincipalRoleMap(faculty).getPrincipalsAndRoles()
+   [('waeup.PortalManager', 'alice', PermissionSetting: Allow)]
+
+But when we delete Alices account from ZODB:
+
+   >>> del root['app']['users']['alice']
+   >>> IPrincipalRoleMap(faculty).getPrincipalsAndRoles()
+   []
+
+the local role has gone and also the site roles have been removed:
+
+   >>> prm = IPrincipalRoleManager(root['app'])
+   >>> [x[0] for x in prm.getRolesForPrincipal('alice')
+   ...      if x[0].startswith('waeup.')]
+   []
+
+
+Logging in via side bar
+=======================
+
+We can access the front page without restrictions:
+
+  >>> browser.open('http://localhost/app')
+  >>> print browser.headers['Status']
+  200 Ok
+
+We have to go to one of the login pages first:
+
+  >>> browser.open('http://localhost/app')
+  >>> browser.getLink('Login').click()
+  >>> print browser.headers['Status']
+  200 Ok
+
+There is a login form on tis page:
+
+  >>> 'form.login' in browser.contents
+  True
+
+  >>> 'form.logout' in browser.contents
+  False
+
+We use this form:
+
+  >>> browser.getControl(name='form.login').value = 'bob'
+  >>> browser.getControl(name='form.password').value = 'invalidpw'
+  >>> browser.getControl('Login').click()
+  >>> 'You entered invalid credentials' in browser.contents
+  True
+
+  >>> browser.getControl(name='form.login').value = 'bob'
+  >>> browser.getControl(name='form.password').value = 'bobsecret'
+  >>> browser.getControl('Login').click()
+
+Now the login form is gone. Instead we have the opportunity to logout:
+
+  >>> 'form.login' in browser.contents
+  False
+
+  >>> logout = browser.getLink('Logout')
+  >>> logout
+  <Link text='Logout' url='http://localhost/app/@@logout'>
+
+The user title is also displayed in the sidebar:
+
+  >>> 'Bob' in browser.contents
+  True
+
+We can also log out afterwards:
+
+  >>> logout.click()
+  >>> print browser.contents
+  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"...
+  ...Login
+  ...
+
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/catalog.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/catalog.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/catalog.txt	(revision 12948)
@@ -0,0 +1,230 @@
+Cataloging Support
+******************
+
+.. module:: waeup.kofa.catalog
+
+Components that support cataloging and searching objects inside a
+Kofa site.
+
+.. :doctest:
+.. :layer: waeup.kofa.testing.KofaUnitTestLayer
+
+The KofaQuery Class
+===================
+
+.. class:: KofaQuery()
+
+   .. attribute:: grok.implements(hurry.query.interfaces.IQuery)
+
+   A `hurry.query.query.Query` compatible query that also supports
+   retrival of plain ``Bree`` result sets as used inside a catalog.
+
+   Like `hurry.query.query.Query` objects, `KofaQuery` is some kind
+   of a meta query or 'compound query' that can give the cataloged
+   objects (or their int ids) matching one or more 'subqueries'.
+
+   This way you can search for objects (or their int ids) that match
+   several criteria at the same time. See ``examples`` section below.
+
+   A singleton instance of this class is also available as global
+   utility.
+
+   .. method:: searchResults(query)
+
+     Get the cataloged objects determined by ``query``.
+
+   .. method:: apply(query)
+
+     Get the list of int ids (a `BTree` result set) for objects
+     determined by ``query``.
+
+     The list of int ids is less expensive to compute than the
+     complete search results and sufficient, for instance, when you
+     only need the number of objects that match a query and not the
+     objects themselves.
+
+Examples
+========
+
+Getting a general query object
+------------------------------
+
+We can get a KofaQuery object by asking for an unnamed global utility
+implementing `hurry.query.interfaces.IQuery`:
+
+    >>> from hurry.query.interfaces import IQuery
+    >>> from zope.component import getUtility
+    >>> q = getUtility(IQuery)
+    >>> q
+    <waeup.kofa.catalog.KofaQuery object at 0x...>
+
+This query can get 'subqueries' and delivers the objects found or
+their ids. To show this we have to setup a catalog with some entries.
+
+
+Setting up a catalog and feeding it
+-----------------------------------
+
+    >>> from zope.catalog.interfaces import ICatalog
+    >>> from zope.catalog.catalog import Catalog
+    >>> mycat = Catalog()
+
+We register this catalog with the component architechture as a utility
+named 'mycatalog':
+
+    >>> from zope.component import provideUtility
+    >>> provideUtility(mycat, ICatalog, 'mycatalog')
+
+We setup a special content type whose instances we will catalog later:
+
+    >>> from zope.interface import Interface, Attribute, implements
+    >>> from zope.container.contained import Contained
+    >>> class IMammoth(Interface):
+    ...   name = Attribute('name')
+    ...   age = Attribute('age')
+
+    >>> class Mammoth(Contained):
+    ...   implements(IMammoth)
+    ...   def __init__(self, name, age):
+    ...     self.name = name
+    ...     self.age = age
+    ...   def __cmp__(self, other):
+    ...     return cmp(self.name, other.name)
+
+By including the __cmp__ method we make sure search results can be
+stably sorted.
+
+We also setup a `zope.intid.interfaces.IIntIds` utility. This is not
+necessary for plain catalogs, but when we want to use KofaQuery (or
+`hurry.query.query.Query` objects), as to get a unique mapping from
+objects (stored in ZODB) to integer numbers (stored in catalogs),
+these query objects lookup a global IIntIds utiliy:
+
+    >>> from zope import interface
+    >>> import zope.intid.interfaces
+    >>> class DummyIntId(object):
+    ...     interface.implements(zope.intid.interfaces.IIntIds)
+    ...     MARKER = '__dummy_int_id__'
+    ...     def __init__(self):
+    ...         self.counter = 0
+    ...         self.data = {}
+    ...     def register(self, obj):
+    ...         intid = getattr(obj, self.MARKER, None)
+    ...         if intid is None:
+    ...             setattr(obj, self.MARKER, self.counter)
+    ...             self.data[self.counter] = obj
+    ...             intid = self.counter
+    ...             self.counter += 1
+    ...         return intid
+    ...     def getObject(self, intid):
+    ...         return self.data[intid]
+    ...     def __iter__(self):
+    ...         return iter(self.data)
+    >>> intid = DummyIntId()
+    >>> from zope.component import provideUtility
+    >>> provideUtility(intid, zope.intid.interfaces.IIntIds)
+
+Now we can catalog some mammoths. Here we create a herd and catalog
+each item of it:
+
+    >>> from zope.catalog.field import FieldIndex
+    >>> mycat['mammoth_name'] = FieldIndex('name', IMammoth)
+    >>> mycat['mammoth_age'] = FieldIndex('age', IMammoth)
+
+    >>> herd = [
+    ...   Mammoth(name='Fred', age=33),
+    ...   Mammoth(name='Hank', age=30),
+    ...   Mammoth(name='Wilma', age=28),
+    ... ]
+
+    >>> for mammoth in herd:
+    ...   mycat.index_doc(intid.register(mammoth), mammoth)
+
+
+Searching for result sets
+-------------------------
+
+Finally we can perform queries:
+
+    >>> from hurry.query import Eq
+    >>> from zope.component import getUtility
+    >>> subquery1 = Eq(('mycatalog', 'mammoth_name'), 'Fred')
+
+The latter means: search for objects whose name is ``'Fred'`` in the
+``mammoth_name`` index of a catalog registered as a utility named
+``mycatalog``.
+
+    >>> from hurry.query import Between
+    >>> subquery2 = Between(('mycatalog', 'mammoth_age'), 30, 33)
+
+This means: ask for objects cataloged in an index named 'mammoth_age',
+whose cataloged value is between 30 and 33 (including this values).
+
+    >>> r1 = q.apply(subquery2)
+    >>> r1
+    IFSet([0, 1])
+
+Using ``apply()`` above, we get a set of values stored in an
+``IFBTree``:
+
+    >>> type(r1)
+    <type 'BTrees.IFBTree.IFSet'>
+
+``IFBTree`` objects implement a rather efficient integer to float
+mapping where also integers are allowed as values. For each object
+found (i.e. mammoths whose age is between 30 and 33), we get the
+number of its entry.
+
+To get the real object, we can use intids here, because we setup an
+appropriate IIntIds utility before:
+
+    >>> [intid.getObject(x).name for x in r1]
+    ['Fred', 'Hank']
+
+We can (and should) also use the `searchResults()` method explained
+below to do that.
+
+Retrieving BTree sets can, however, make sense, if you want to know
+only the number of results for a particular query or whether there are
+results at all in a more efficient way:
+
+    >>> len(r1)
+    2
+
+Searching for objects
+---------------------
+
+Very often we don't want to know the catalog-internal 'ids' of
+searched objects but the objects themselves.
+
+This can be done by using the ``searchResults`` method of
+``KofaQuery``:
+
+    >>> r2 = q.searchResults(subquery1)
+    >>> r2
+    <zope.catalog.catalog.ResultSet instance at 0x...>
+
+    >>> list(r2)
+    [<Mammoth object at 0x...>]
+
+We got one result item, we can immediately ask for further infos. To
+access a result item by its index number, we have to turn the
+ResultSet into an ordinary list before:
+
+    >>> entry = list(r2)[0]
+    >>> entry.name, entry.age
+    ('Fred', 33)
+
+We can also use ``subquery2`` as above:
+
+    >>> r3 = q.searchResults(subquery2)
+    >>> [(x.name, x.age) for x in r3]
+    [('Fred', 33), ('Hank', 30)]
+
+or use both queries at once:
+
+    >>> r4 = q.searchResults(subquery1 & subquery2)
+    >>> [(x.name, x.age) for x in r4]
+    [('Fred', 33)]
+
+which will give us, of course, the same result set as with subquery1.
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/datacenter.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/datacenter.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/datacenter.txt	(revision 12948)
@@ -0,0 +1,290 @@
+Data Center
+***********
+
+The Kofa data center cares for managing CSV files and importing then.
+
+.. :doctest:
+.. :layer: waeup.kofa.testing.KofaUnitTestLayer
+
+Creating a data center
+======================
+
+A data center can be created easily:
+
+    >>> from waeup.kofa.datacenter import DataCenter
+    >>> mydatacenter = DataCenter()
+    >>> mydatacenter
+    <waeup.kofa.datacenter.DataCenter object at 0x...>
+
+Each data center has a location in file system where files are stored:
+
+    >>> storagepath = mydatacenter.storage
+    >>> storagepath
+    '/tmp/tmp...'
+
+Beside other things it provides a location to put data of deleted
+items into:
+
+    >>> import os
+    >>> del_path = mydatacenter.deleted_path
+    >>> os.path.isdir(del_path)
+    True
+
+Overall it complies with the `IDataCenter` interface:
+
+    >>> from zope.interface import verify
+    >>> from waeup.kofa.interfaces import IDataCenter
+    >>> verify.verifyObject(IDataCenter, DataCenter() )
+    True
+
+    >>> verify.verifyClass(IDataCenter, DataCenter)
+    True
+
+Managing the storage path
+=========================
+
+We can set another storage path:
+
+    >>> import os
+    >>> os.mkdir('newlocation')
+    >>> newpath = os.path.abspath('newlocation')
+    >>> mydatacenter.setStoragePath(newpath)
+    []
+
+The result here is a list of filenames, that could not be
+copied. Luckily, this list is empty.
+
+When we set a new storage path, we can tell to move all files in the
+old location to the new one. To see this feature in action, we first
+have to put a file into the old location:
+
+    >>> open(os.path.join(newpath, 'myfile.txt'), 'wb').write('hello')
+
+Now we can set a new location and the file will be copied:
+
+    >>> verynewpath = os.path.abspath('verynewlocation')
+    >>> os.mkdir(verynewpath)
+
+    >>> mydatacenter.setStoragePath(verynewpath, move=True)
+    []
+
+    >>> storagepath = mydatacenter.storage
+    >>> 'myfile.txt' in os.listdir(verynewpath)
+    True
+
+We remove the created file to have a clean testing environment for
+upcoming examples:
+
+    >>> os.unlink(os.path.join(storagepath, 'myfile.txt'))
+
+Uploading files
+===============
+
+We can get a list of files stored in that location:
+
+    >>> mydatacenter.getPendingFiles()
+    []
+
+Let's put some file in the storage:
+
+    >>> import os
+    >>> filepath = os.path.join(storagepath, 'data.csv')
+    >>> open(filepath, 'wb').write('Some Content\n')
+
+Now we can find a file:
+
+    >>> mydatacenter.getPendingFiles()
+    [<waeup.kofa.datacenter.DataCenterFile object at 0x...>]
+
+As we can see, the actual file is wrapped by a convenience wrapper,
+that enables us to fetch some data about the file. The data returned
+is formatted in strings, so that it can easily be put into output
+pages:
+
+    >>> datafile = mydatacenter.getPendingFiles()[0]
+    >>> datafile.getSize()
+    '13 bytes'
+
+    >>> datafile.getDate() # Nearly current datetime...
+    '...'
+
+Clean up:
+
+    >>> import shutil
+    >>> shutil.rmtree(newpath)
+    >>> shutil.rmtree(verynewpath)
+
+
+Distributing processed files
+============================
+
+When files were processed by a batch processor, we can put the
+resulting files into desired destinations.
+
+We recreate the datacenter root in case it is missing:
+
+    >>> import os
+    >>> dc_root = mydatacenter.storage
+    >>> fin_dir = os.path.join(dc_root, 'finished')
+    >>> unfin_dir = os.path.join(dc_root, 'unfinished')
+
+    >>> def recreate_dc_storage():
+    ...   if os.path.exists(dc_root):
+    ...     shutil.rmtree(dc_root)
+    ...   os.mkdir(dc_root)
+    ...   mydatacenter.setStoragePath(mydatacenter.storage)
+    >>> recreate_dc_storage()
+
+We define a function that creates a set of faked result files:
+
+    >>> import os
+    >>> import tempfile
+    >>> def create_fake_results(source_basename, create_pending=True):
+    ...   tmp_dir = tempfile.mkdtemp()
+    ...   src = os.path.join(dc_root, source_basename)
+    ...   pending_src = None
+    ...   if create_pending:
+    ...     pending_src = os.path.join(tmp_dir, 'mypendingsource.csv')
+    ...   finished_src = os.path.join(tmp_dir, 'myfinishedsource.csv')
+    ...   for path in (src, pending_src, finished_src):
+    ...     if path is not None:
+    ...       open(path, 'wb').write('blah')
+    ...   return tmp_dir, src, finished_src, pending_src
+
+Now we can create the set of result files, that typically come after a
+successful processing of a regular source:
+
+Now we can try to distribute those files. Let's start with a source
+file, that was processed successfully:
+
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.csv', create_pending=False)
+    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
+    ...                            pending_src, mode='create')
+    >>> sorted(os.listdir(dc_root))
+    ['deleted', 'finished', 'logs', 'unfinished']
+
+    >>> sorted(os.listdir(fin_dir))
+    ['mysource.create.finished.csv', 'mysource.csv']
+
+    >>> sorted(os.listdir(unfin_dir))
+    []
+
+The created dir will be removed for us by the datacenter. This way we
+can assured, that less temporary dirs are left hanging around:
+
+    >>> os.path.exists(tmp_dir)
+    False
+
+The root dir is empty, while the original file and the file containing
+all processed data were moved to'finished/'.
+
+Now we restart, but this time we fake an erranous action:
+
+    >>> recreate_dc_storage()
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.csv')
+    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
+    ...                                 pending_src, mode='create')
+    >>> sorted(os.listdir(dc_root))
+    ['deleted', 'finished', 'logs', 'mysource.create.pending.csv', 'unfinished']
+
+    >>> sorted(os.listdir(fin_dir))
+    ['mysource.create.finished.csv']
+
+    >>> sorted(os.listdir(unfin_dir))
+    ['mysource.csv']
+
+While the original source was moved to the 'unfinished' dir, the
+pending file went to the root and the set of already processed items
+are stored in finished/.
+
+We fake processing the pending file and assume that everything went
+well this time:
+
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.create.pending.csv', create_pending=False)
+    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
+    ...                                 pending_src, mode='create')
+
+    >>> sorted(os.listdir(dc_root))
+    ['deleted', 'finished', 'logs', 'unfinished']
+
+    >>> sorted(os.listdir(fin_dir))
+    ['mysource.create.finished.csv', 'mysource.csv']
+
+    >>> sorted(os.listdir(unfin_dir))
+    []
+
+The result is the same as in the first case shown above.
+
+We restart again, but this time we fake several non-working imports in
+a row.
+
+We start with a faulty start-import:
+
+    >>> recreate_dc_storage()
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.csv')
+    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
+    ...                                 pending_src, mode='create')
+
+We try to process the pending file, which fails again:
+
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.create.pending.csv')
+    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
+    ...                                 pending_src, mode='create')
+
+We try to process the new pending file:
+
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.create.pending.csv')
+    >>> mydatacenter.distProcessedFiles(False, src, finished_src,
+    ...                                 pending_src, mode='create')
+
+    >>> sorted(os.listdir(dc_root))
+    ['deleted', 'finished', 'logs', 'mysource.create.pending.csv', 'unfinished']
+
+    >>> sorted(os.listdir(fin_dir))
+    ['mysource.create.finished.csv']
+
+    >>> sorted(os.listdir(unfin_dir))
+    ['mysource.csv']
+
+Finally, we process the pending file and everything works:
+
+    >>> tmp_dir, src, finished_src, pending_src = create_fake_results(
+    ...  'mysource.create.pending.csv', create_pending=False)
+    >>> mydatacenter.distProcessedFiles(True, src, finished_src,
+    ...                                 pending_src, mode='create')
+
+    >>> sorted(os.listdir(dc_root))
+    ['deleted', 'finished', 'logs', 'unfinished']
+
+    >>> sorted(os.listdir(fin_dir))
+    ['mysource.create.finished.csv', 'mysource.csv']
+
+    >>> sorted(os.listdir(unfin_dir))
+    []
+
+The root dir is empty (contains no input files) and only the files in
+finished-subdirectory remain.
+
+
+We can get a list of imported files stored in the finished subfolder:
+
+    >>> mydatacenter.getFinishedFiles()
+    [<waeup.kofa.datacenter.DataCenterFile object at ...>]
+
+    >>> datafile = mydatacenter.getFinishedFiles()[0]
+    >>> datafile.getSize()
+    '2 bytes'
+
+    >>> datafile.getDate() # Nearly current datetime...
+    '...'
+
+
+Clean up:
+
+    >>> shutil.rmtree(verynewpath)
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/permissions.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/permissions.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/permissions.txt	(revision 12948)
@@ -0,0 +1,117 @@
+Permissions and Roles
+*********************
+
+Permissions and roles used in a Kofa portal.
+
+.. :doctest:
+.. :layer: waeup.kofa.testing.KofaUnitTestLayer
+
+Convenience Functions
+=====================
+
+:mod:`waeup.kofa` offers some convenience functions to handle security
+roles.
+
+:func:`get_all_roles`
+---------------------
+
+Gives us all roles defined in Kofa. We get tuples of
+kind
+
+  ``(<ROLE-NAME>, <ROLE>)``
+
+where ``<ROLE-NAME>`` is the name under which a role was registered
+with the ZCA (a string) and ``<ROLE>`` is the real role object.
+
+    >>> from waeup.kofa.permissions import get_all_roles
+    >>> get_all_roles()
+    <generator object...at 0x...>
+
+    >>> sorted(list(get_all_roles()))
+    [(u'waeup.ACManager', <waeup.kofa.permissions.ACManager object at 0x...]
+
+:func:`get_waeup_roles`
+-----------------------
+
+Gives us all roles, except the Kofa specific roles. We can get a list
+with or without local roles:
+
+    >>> from waeup.kofa.permissions import get_waeup_roles
+    >>> len(list(get_waeup_roles()))
+    26
+
+    >>> len(list(get_waeup_roles(also_local=True)))
+    47
+
+
+:func:`get_waeup_role_names`
+----------------------------
+
+We can get all role names defined in Kofa (except 'local'
+roles that are meant not to be assigned globally):
+
+    >>> from waeup.kofa.permissions import get_waeup_role_names
+    >>> list(get_waeup_role_names())
+    [u'waeup.ACManager',
+     u'waeup.AcademicsManager',
+     u'waeup.AcademicsOfficer',
+     u'waeup.AccommodationOfficer',
+     u'waeup.Applicant',
+     u'waeup.ApplicationsManager',
+     u'waeup.ApplicationsOfficer',
+     u'waeup.BursaryOfficer',
+     u'waeup.DataCenterManager',
+     u'waeup.DocumentsManager',
+     u'waeup.DocumentsOfficer',
+     u'waeup.ExportManager',
+     u'waeup.ImportManager',
+     u'waeup.PortalManager',
+     u'waeup.ReportsManager',
+     u'waeup.ReportsOfficer',
+     u'waeup.Student',
+     u'waeup.StudentImpersonator',
+     u'waeup.StudentsClearanceOfficer',
+     u'waeup.StudentsCourseAdviser',
+     u'waeup.StudentsManager',
+     u'waeup.StudentsOfficer',
+     u'waeup.TranscriptOfficer',
+     u'waeup.UsersManager',
+     u'waeup.WorkflowManager',
+     u'waeup.xmlrpcusers1']
+
+:func:`get_users_with_local_roles`
+----------------------------------
+
+We can get all users and their roles for a certain context
+object. This even works for objects that cannot have local roles as
+they are not stored in the ZODB:
+
+    >>> from waeup.kofa.permissions import get_users_with_local_roles
+    >>> mycontext = object()
+    >>> people_and_roles = get_users_with_local_roles(mycontext)
+    >>> people_and_roles
+    <generator object...at 0x...>
+
+In this case, the result is empty:
+
+    >>> people_and_roles = list(people_and_roles)
+    >>> people_and_roles
+    []
+
+:func:`get_users_with_role`
+---------------------------
+
+We can get all users with a specific role for a certain context
+object:
+
+    >>> from waeup.kofa.permissions import get_users_with_role
+    >>> mycontext = object()
+    >>> people = get_users_with_role('waeup.portalManager', mycontext)
+    >>> people
+    <generator object...at 0x...>
+
+In this case, the result is empty:
+
+    >>> people = list(people)
+    >>> people
+    []
Index: /main/waeup.kofa/trunk/src/waeup/kofa/doctests/userscontainer.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/doctests/userscontainer.txt	(revision 12948)
+++ /main/waeup.kofa/trunk/src/waeup/kofa/doctests/userscontainer.txt	(revision 12948)
@@ -0,0 +1,80 @@
+Users Container
+***************
+
+.. :doctest:
+.. :layer: waeup.kofa.testing.KofaUnitTestLayer
+
+Before we can start, we need some password managers available:
+
+    >>> from zope.app.authentication.placelesssetup import (
+    ...   PlacelessSetup)
+    >>> PlacelessSetup().setUp()
+
+We can create a user container which will hold the useraccounts for
+us:
+
+    >>> from waeup.kofa.userscontainer import UsersContainer
+    >>> myusers = UsersContainer()
+
+We can add users, just by passing a name, a password, and (optionally)
+a title and a description:
+
+    >>> myusers.addUser('bob', 'bobssecret')
+
+Now, Bob is in the container:
+
+    >>> list(myusers)
+    [u'bob']
+
+We can get Bob's account:
+
+    >>> bob = myusers['bob']
+    >>> bob
+    <waeup.kofa.authentication.Account object at 0x...>
+
+    >>> bob.name
+    'bob'
+
+    >>> bob.title
+    'bob'
+
+    >>> bob.description
+
+
+As we did not give a title, the name was taken instead
+for title (but not for description). The password, however, is stored encoded:
+
+    >>> bob.password
+    '{SSHA}...'
+
+We can export user accounts:
+
+    >>> from waeup.kofa.userscontainer import UserExporter
+    >>> import os
+    >>> import tempfile
+    >>> workdir = tempfile.mkdtemp()
+    >>> outfile = os.path.join(workdir, 'myoutput.csv')
+    >>> exporter = UserExporter()
+    >>> site = {'users':myusers}
+    >>> exporter.export_all(site, outfile)
+    >>> result = open(outfile, 'rb').read()
+    >>> 'name,title,public_name,description,email,phone,roles,local_roles,password' in result
+    True
+    >>> '{SSHA}' in result
+    True
+
+We can delete users:
+
+    >>> myusers.delUser('alice')
+
+The container won't complain, although there is no ``alice`` stored
+yet. But we can really delete users:
+
+    >>> myusers.delUser('bob')
+    >>> list(myusers)
+    []
+
+Clean up:
+
+    >>> import shutil
+    >>> shutil.rmtree(workdir)
Index: in/waeup.kofa/trunk/src/waeup/kofa/permissions.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/permissions.txt	(revision 12947)
+++ 	(revision )
@@ -1,117 +1,0 @@
-Permissions and Roles
-*********************
-
-Permissions and roles used in a Kofa portal.
-
-.. :doctest:
-.. :layer: waeup.kofa.testing.KofaUnitTestLayer
-
-Convenience Functions
-=====================
-
-:mod:`waeup.kofa` offers some convenience functions to handle security
-roles.
-
-:func:`get_all_roles`
----------------------
-
-Gives us all roles defined in Kofa. We get tuples of
-kind
-
-  ``(<ROLE-NAME>, <ROLE>)``
-
-where ``<ROLE-NAME>`` is the name under which a role was registered
-with the ZCA (a string) and ``<ROLE>`` is the real role object.
-
-    >>> from waeup.kofa.permissions import get_all_roles
-    >>> get_all_roles()
-    <generator object...at 0x...>
-
-    >>> sorted(list(get_all_roles()))
-    [(u'waeup.ACManager', <waeup.kofa.permissions.ACManager object at 0x...]
-
-:func:`get_waeup_roles`
------------------------
-
-Gives us all roles, except the Kofa specific roles. We can get a list
-with or without local roles:
-
-    >>> from waeup.kofa.permissions import get_waeup_roles
-    >>> len(list(get_waeup_roles()))
-    26
-
-    >>> len(list(get_waeup_roles(also_local=True)))
-    47
-
-
-:func:`get_waeup_role_names`
-----------------------------
-
-We can get all role names defined in Kofa (except 'local'
-roles that are meant not to be assigned globally):
-
-    >>> from waeup.kofa.permissions import get_waeup_role_names
-    >>> list(get_waeup_role_names())
-    [u'waeup.ACManager',
-     u'waeup.AcademicsManager',
-     u'waeup.AcademicsOfficer',
-     u'waeup.AccommodationOfficer',
-     u'waeup.Applicant',
-     u'waeup.ApplicationsManager',
-     u'waeup.ApplicationsOfficer',
-     u'waeup.BursaryOfficer',
-     u'waeup.DataCenterManager',
-     u'waeup.DocumentsManager',
-     u'waeup.DocumentsOfficer',
-     u'waeup.ExportManager',
-     u'waeup.ImportManager',
-     u'waeup.PortalManager',
-     u'waeup.ReportsManager',
-     u'waeup.ReportsOfficer',
-     u'waeup.Student',
-     u'waeup.StudentImpersonator',
-     u'waeup.StudentsClearanceOfficer',
-     u'waeup.StudentsCourseAdviser',
-     u'waeup.StudentsManager',
-     u'waeup.StudentsOfficer',
-     u'waeup.TranscriptOfficer',
-     u'waeup.UsersManager',
-     u'waeup.WorkflowManager',
-     u'waeup.xmlrpcusers1']
-
-:func:`get_users_with_local_roles`
-----------------------------------
-
-We can get all users and their roles for a certain context
-object. This even works for objects that cannot have local roles as
-they are not stored in the ZODB:
-
-    >>> from waeup.kofa.permissions import get_users_with_local_roles
-    >>> mycontext = object()
-    >>> people_and_roles = get_users_with_local_roles(mycontext)
-    >>> people_and_roles
-    <generator object...at 0x...>
-
-In this case, the result is empty:
-
-    >>> people_and_roles = list(people_and_roles)
-    >>> people_and_roles
-    []
-
-:func:`get_users_with_role`
----------------------------
-
-We can get all users with a specific role for a certain context
-object:
-
-    >>> from waeup.kofa.permissions import get_users_with_role
-    >>> mycontext = object()
-    >>> people = get_users_with_role('waeup.portalManager', mycontext)
-    >>> people
-    <generator object...at 0x...>
-
-In this case, the result is empty:
-
-    >>> people = list(people)
-    >>> people
-    []
Index: in/waeup.kofa/trunk/src/waeup/kofa/userscontainer.txt
===================================================================
--- /main/waeup.kofa/trunk/src/waeup/kofa/userscontainer.txt	(revision 12947)
+++ 	(revision )
@@ -1,80 +1,0 @@
-Users Container
-***************
-
-.. :doctest:
-.. :layer: waeup.kofa.testing.KofaUnitTestLayer
-
-Before we can start, we need some password managers available:
-
-    >>> from zope.app.authentication.placelesssetup import (
-    ...   PlacelessSetup)
-    >>> PlacelessSetup().setUp()
-
-We can create a user container which will hold the useraccounts for
-us:
-
-    >>> from waeup.kofa.userscontainer import UsersContainer
-    >>> myusers = UsersContainer()
-
-We can add users, just by passing a name, a password, and (optionally)
-a title and a description:
-
-    >>> myusers.addUser('bob', 'bobssecret')
-
-Now, Bob is in the container:
-
-    >>> list(myusers)
-    [u'bob']
-
-We can get Bob's account:
-
-    >>> bob = myusers['bob']
-    >>> bob
-    <waeup.kofa.authentication.Account object at 0x...>
-
-    >>> bob.name
-    'bob'
-
-    >>> bob.title
-    'bob'
-
-    >>> bob.description
-
-
-As we did not give a title, the name was taken instead
-for title (but not for description). The password, however, is stored encoded:
-
-    >>> bob.password
-    '{SSHA}...'
-
-We can export user accounts:
-
-    >>> from waeup.kofa.userscontainer import UserExporter
-    >>> import os
-    >>> import tempfile
-    >>> workdir = tempfile.mkdtemp()
-    >>> outfile = os.path.join(workdir, 'myoutput.csv')
-    >>> exporter = UserExporter()
-    >>> site = {'users':myusers}
-    >>> exporter.export_all(site, outfile)
-    >>> result = open(outfile, 'rb').read()
-    >>> 'name,title,public_name,description,email,phone,roles,local_roles,password' in result
-    True
-    >>> '{SSHA}' in result
-    True
-
-We can delete users:
-
-    >>> myusers.delUser('alice')
-
-The container won't complain, although there is no ``alice`` stored
-yet. But we can really delete users:
-
-    >>> myusers.delUser('bob')
-    >>> list(myusers)
-    []
-
-Clean up:
-
-    >>> import shutil
-    >>> shutil.rmtree(workdir)
