Changeset 4221 for waeup

Timestamp:

8 Jun 2009, 16:56:05 (15 years ago)

Author:

uli

Message:

Update tests.

File:

• waeup/branches/ulif-rewrite/src/waeup/utils/importexport.txt (modified) (1 diff)

waeup/branches/ulif-rewrite/src/waeup/utils/importexport.txt

@@ -149 +149 @@
 data types like faculties, students, and the like.
 
-Obtaining importers
--------------------
-
-Before we actually do real imports, we have to provide so-called
-importers, that will do the job for us. Let's see how that works.
-
-We first create an object into that we can import something:
-
-    >>> import grok
-    >>> from waeup.interfaces import IWAeUPObject
-    >>> class MyContainer(grok.Container):
-    ...   #grok.implements(IWAeUPObject)
-    ...   def foo(self):
-    ...     print "Boo!"
-
-We get a suitable importer for certain objects by asking for an
-adapter to `IWAeUPCSVImporter`:
-
+For our purposes we define CSV imports as tuples, containing a CSV
+source and a CSV receiver:
+
+  ``CSV Import := <CSVSource, CSVReceiver>``
+
+CSV sources
+-----------
+
+CSV sources build a plugin framework inside the WAeUP portal (that
+might be factored out one day). See `waeup.csvfile` to learn more
+about that.
+
+A `waeup.csvfile.CSVFile` plugin cares for a special kind of CSV file.
+
+To make clear what that means, we start by creating a simple CSV
+file.
+
+CSVFile objects are abstractions of simple .csv-files. But as not
+every CSV file contains the same kind of data, we can create different
+flavours of it, for instance to import departments or faculties. As
+each kind of import needs different kind of data, we have different
+kinds of CSV sources.
+
+CSV sources can check their associated files for consistency,
+importability, etc.
+
+We create a simple CSV file with cave data:
+
+    >>> open('mycavedata.csv', 'wb').write(
+    ... """size,owner
+    ... 42,Manfred
+    ... """)
+
+We have a very simple CSVFile available, which is registered as an
+adapter for strings. To make this work, the string should be a path to
+a file:
+
+    >>> from waeup.csvfile import getCSVFile
+    >>> mysource = getCSVFile('mycavedata.csv')
+    >>> mysource
+    <waeup.csvfile.csvfile.CSVFile object at 0x...>
+
+We define caves like this:
+
+    >>> class Cave(object):
+    ...   def __init__(self, size, owner):
+    ...     self.size = size
+    ...     self.owner = owner
+    ...   def __repr__(self):
+    ...     return '<Cave object [size: %s, owner: %s]>' % (
+    ...       self.size, self.owner)
+
+Let's assume, we have a container for caves:
+
+    >>> from zope.interface import Interface
+    >>> class ICaveContainer(Interface):
+    ...   """A container for caves."""
+
+    >>> class CaveContainer(object):
+    ...   grok.implements(ICaveContainer)
+    ...   caves = []
+    ...   def addCave(self, cave):
+    ...     self.caves.append(cave)
+
+    >>> mycaves = CaveContainer()
+
+Now, if we want to import caves from CSV files, we also need an
+importer, that imports data to the container:
+
+    >>> from waeup.csvfile.interfaces import ICSVFile
+    >>> from waeup.utils.importexport import CSVImporter
     >>> from waeup.interfaces import IWAeUPCSVImporter
-
-If, however, we try to create an importer for our container, we won't
-find one right now:
-
-    >>> mycontainer = MyContainer()
-    >>> importer = IWAeUPCSVImporter(mycontainer)
-    Traceback (most recent call last):
-    ...
-    TypeError: ('Could not adapt', <MyContainer ...>, 
-                <InterfaceClass ...IWAeUPCSVImporter>)
-
-The reason is, that there is simply no importer defined for
-`MyContainer` objects.
-
-As an importer must take care for the specific structure of objects
-that receive the data, we must first define an importer for our
-special container class:
-
-    >>> import grok
-    >>> from waeup.utils.importexport import CSVImporter
-    >>> class MyImporter(CSVImporter):
-    ...   grok.context(MyContainer)
-    ...   datatype = u'My Stuff'
-    ...   column_terms = ['col1', 'col2']
-    ...   def doImport(self, filepath, clear_old_data=True,
-    ...                                overwrite=True):
-    ...     print "Data imported!"
-
-This importer derives from `CSVImporter`. The `CSVImporter` base class
-makes our importer automatically an adapter. Therefore we have to give
-the type of object we provide this importer for by using
-`grok.context()`.
-
-Instances of our importer will have an instance variable
-`self.context` available, which will hold the specific object into
-which data should be imported.
-
-The `datatype` string gives a description of the kind of importer we
-define. It might be used by UI components to offer services of this
-importer.
-
-The `column_terms` class variable holds the column headings we expect
-in a received CSV file.
-
-Our importer should also comply with the IWAeUPCSVImporter interface,
-therefore we defined a `doImport` method.
-
-To register our importer with the framework, we have to ``grok`` it
-first. Note, that in normal use this is done automatically on startup
-by the Grok framework:
-
-    >>> grok.testing.grok_component('MyImporter', MyImporter)
+    >>> class CaveImporter(CSVImporter):
+    ...   # Tell, what kinds of objects we connect...
+    ...   grok.adapts(ICSVFile, ICaveContainer)
+    ...   # Tell the world, that we are an importer...
+    ...   grok.provides(IWAeUPCSVImporter)
+    ...   # The datatype can be used by UI components and is required
+    ...   # by the interface...
+    ...   datatype = u'Cave Importer'
+    ...   def doImport(self):
+    ...     # CSVImporter instances have a `csvfile` and a `receiver`
+    ...     # object defined which refer to the CSV file and the container.
+    ...     for row in self.csvfile.getData():
+    ...       cave = Cave(size = row['size'],
+    ...                   owner = row['owner'])
+    ...       self.receiver.addCave(cave) 
+
+
+An CSV importer must be grokked before we can use it. In real life,
+this is normally done on startup automatically:
+
+    >>> grok.testing.grok_component('CaveImporter', CaveImporter)
     True
 
-Now we can get an importer:
-
-    >>> importer = IWAeUPCSVImporter(mycontainer)
-    >>> importer
-    <MyImporter object at 0x...>
-
-This importer is rather useless as it does not do much:
-
-    >>> importer.doImport('nonsense')
-    Data imported!
-
-This is a shameless lie. You might, however, get an idea of how to
-build your own, real importer.
-
-From the baseclass, however, we get some functionality out-of-the-box:
-We can validate files (without importing them):
-
-    >>> open('mydata.csv', 'wb').write(
-    ... """col1,col2
-    ... item1,item2
-    ... """)
-
-    >>> importer.validate('mydata.csv')
-    True
-
-If we deliver a misformatted file, the validator will notice that:
-
-    >>> open('crapdata.csv', 'wb').write("""
-    ... col1,blah
-    ... item1,item2
-    ... """)
-
-    >>> importer.validate('crapdata.csv')
-    Traceback (most recent call last):
-    ...
-    KeyError: 'Validation Error: ... could not be found: col2'
-
-Apparently our input file lacks a ``col2`` column.
-
-The WAeUP portal already comes with a set of better suited
-importers. As they rely pretty much on the data structures they are
-built for, they are normally defined in the approriate code pieces
-where also the data structure itself is handled. So you can find an
-importer for faculty CSV data in the `facultycontainer` module.
+Importers are multi-adapters, so we need to ask for one:
+
+    >>> from zope.component import getMultiAdapter
+    >>> myimporter = getMultiAdapter((mysource, mycaves), IWAeUPCSVImporter)
+
+Now we can finally do the import:
+
+    >>> myimporter.doImport()
+
+The import was really done:
+
+    >>> mycaves.caves
+    [<Cave object [size: 42, owner: Manfred]>]
+
+**Important note**:
+
+  This importer adapts ICSVFile and ICaveContainer, which means that
+  the type of data receiver is specified correctly. For the CSV
+  source, however, the importer cannot be sure to find a column
+  ``size`` or ``owner`` in the file.
+
+  This can be prevented, by defining a more special CSVFile type and
+  adapting this instead of ICSVData.
+
+  In the `waeup.csvfile` subpackage's README it is shown, how this can
+  be accomplished.
+
+  Summing up we would define an ICaveCSVFile type, provide an
+  appropriate CSV file wrapper and then let our importer adapt::
+
+    (ICaveCSVFile, ICaveContainer)
+
+**Another note**:
+
+  The importer we defined above does not support the ``clear_old_data``
+  and ``overwrite`` keywords as would be required by the
+  IWAeUPImporter interface.
+
+  In real life this keywords should be supported.
 
 Clean up:
 
-    >>> import os
-    >>> os.unlink('crapdata.csv')
-    >>> os.unlink('mydata.csv')
-
+   >>> import os
+   >>> os.unlink('mycavedata.csv')