Ignore:
Timestamp:
15 Apr 2015, 14:28:46 (10 years ago)
Author:
Henrik Bettermann
Message:

Documentation work in progress.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • main/waeup.kofa/trunk/docs/source/userdocs/datacenter.rst

    r12829 r12857  
    33Data Center
    44***********
     5
     6.. contents::
     7
     8Introduction
     9============
     10
     11Most web portals store their data in a relational database like PostgreSQL, MySQL or Oracle. A relational database is organized in tables of rows and columns, with a unique key for each row. Each data entity gets its own table. Rows in tables can be linked to rows in other tables by storing the unique key of the row to which it should be linked. This sounds quite simple. Many computer users are familiar with this kind of data storage because they are used to spreadsheet programmes like Excel oder Calc which also organize data in tables.
     12
     13Kofa's persistent data are stored in a native object database designed for the Python programming language, the so-called ZODB_. An object database stores objects with attributes and not records as rows with columns in tables. These persistent objects can hold any kind of information in attributes and must not adhere to a specific schema like records in tables of a relational database.
     14
     15The ZODB_ also supports a hierarchical, treelike storage of objects. Objects can contain other objects if they are declared as containers. Objects are stored like folders and files in a filesystem. This makes the object handling very fast and transparent because we can access objects, or more precisely views of objects, by indicating their path in the database, i.e. by traversing the database tree to find an object. Furthermore, we are accessing the views of objects through a web browser by entering a URL (Uniform Resource Locator). This publication path corresponds more or less to the traversal path of our objects. In Kofa the path always contains the object identifiers of all objects which are passed when traversing the database tree. Example:
     16
     17https://kofa-demo.waeup.org/students/K1000000/studycourse/100/DCO
     18
     19is the URL which requests a display view of a course ticket with id 'DCO'. This object is stored in a study level container object with id '100', stored in a study course container object with id 'studycourse', stored in the student container object with id 'K1000000', stored in the students root container, stored in the root container of the application, stored in the root of the database itself.
     20
     21This kind of storage requires that each object gets a unique object identifier (object id) within its container. The id string is visible in the browser address bar. Though it's technically possible for ids to contain spaces or slashes we do not allow these kinds of special characters in object ids to facilitate the readability of URLs.
     22
     23Administrators of web portals, which store their data in relational databases, are used to getting direct access to the portal's database. There are even tools to handle the administration of these databases over the Internet, like phpMyAdmin or phpPgAdmin to handle MySQL or PostgreSQL databases respectively. These user interfaces bypass the portals' user interfaces and give direct access to the database. They allow to easily import or export (dump) data tables or the entire database structure into CSV or SQL files. What at first sight appears to be very helpful and administration-friendly proves to be very dangerous on closer inspection. Data structures can be easily damaged or destroyed, or data can be easily manipulated by circumventing the portal's security machinery or logging system. Kofa does not provide any external user interface to access the ZODB directly, neither for viewing nor for editing data. This includes also the export and import of sets of data. Exports and imports are handled via the Kofa user interface itself. This is called batch processing which means either producing CSV files (comma-separated values) from portal data (export) or processing CSV files in order to add, update or remove portal data (import). Main premise of Kofa's batch processing technology is that the data stored in the ZODB can be specifically backed up and restored by exporting and importing data. But that's not all. Batch processors can do much more. They are an integral part of the student registration management.
     24
     25Data Export
     26===========
     27
     28Regular data exporters (1) collect objects from specific containers, (2) iterate over the collected objects, (3) extract and mangle information from each object, (4) write the information of each object into a row of a CSV file and (5) finally provide the file for download. The CSV file is neither stored in the database nor archived in the filesystem. (3) and (4) is a flattening of the hierarchical data structure, i.e. a mapping of objects to flat relational data stored in a CSV table. The extracted information must not necessarily be based only on static attributes of the collected object. The data, finally stored in the CSV file, can also be derived from parent or child objects, or dynamically computed by the object's methods and property attributes. These methods and properties can retrieve information from everywhere in the portal's database.
     29
     30.. autoclass:: waeup.kofa.university.export.FacultyExporter()
     31  :noindex:
     32
     33  .. autoattribute:: waeup.kofa.university.export.FacultyExporter.fields
     34  .. autoattribute:: waeup.kofa.university.export.FacultyExporter.title
     35
     36  .. automethod:: waeup.kofa.university.export.FacultyExporter.mangle_value()
     37
     38
     39
     40
     41Data Import
     42===========
     43
     44Logging
     45=======
     46
     47
     48
     49
     50
     51
     52
     53.. _ZODB: http://www.zodb.org/
Note: See TracChangeset for help on using the changeset viewer.