| 1 | .. _datacenter_intro: |
|---|
| 2 | |
|---|
| 3 | Introduction |
|---|
| 4 | ************ |
|---|
| 5 | |
|---|
| 6 | .. _object_database: |
|---|
| 7 | |
|---|
| 8 | The Object Database |
|---|
| 9 | =================== |
|---|
| 10 | |
|---|
| 11 | Most web portals store their data in a relational database like |
|---|
| 12 | PostgreSQL, MySQL or Oracle. A relational database is organized in |
|---|
| 13 | tables of rows and columns, with a unique key for each row. Each data |
|---|
| 14 | entity gets its own table. Rows in tables can be linked to rows in |
|---|
| 15 | other tables by storing the unique key of the row to which it should |
|---|
| 16 | be linked. This sounds quite simple. Many computer users are familiar |
|---|
| 17 | with this kind of data storage because they are used to spreadsheet |
|---|
| 18 | programmes like Excel oder Calc which also organize data in tables. |
|---|
| 19 | Kofa's persistent data are stored in a native object database designed |
|---|
| 20 | for the Python programming language, the so-called ZODB_. An object |
|---|
| 21 | database stores objects with attributes and not records as rows with |
|---|
| 22 | columns in tables. These persistent objects can hold any kind of |
|---|
| 23 | information in attributes and must not adhere to a specific schema |
|---|
| 24 | like records in tables of a relational database. |
|---|
| 25 | |
|---|
| 26 | The ZODB_ also supports a hierarchical, treelike storage of objects. |
|---|
| 27 | Objects can contain other objects if they are declared as containers. |
|---|
| 28 | Objects are stored like folders and files in a filesystem. This makes |
|---|
| 29 | the object handling very fast and transparent because we can access |
|---|
| 30 | objects, or more precisely views of objects, by indicating their path |
|---|
| 31 | in the database, i.e. by traversing the database tree to the object's |
|---|
| 32 | location. Furthermore, we are accessing the views of objects through a |
|---|
| 33 | web browser by entering a URL (Uniform Resource Locator). This |
|---|
| 34 | publication path corresponds more or less to the traversal path of our |
|---|
| 35 | objects. In Kofa the path always contains the object identifiers of |
|---|
| 36 | all objects which are passed when traversing the database tree. |
|---|
| 37 | Example: |
|---|
| 38 | |
|---|
| 39 | https://kofa-demo.waeup.org/students/K1000000/studycourse/100/DCO |
|---|
| 40 | |
|---|
| 41 | is the URL which requests a display view of a course ticket with id |
|---|
| 42 | ``DCO``. This object is stored in a study level container object with |
|---|
| 43 | id ``100``, stored in a study course container object with id |
|---|
| 44 | ``studycourse``, stored in the student container object with id |
|---|
| 45 | ``K1000000``, stored in the students root container, stored in the |
|---|
| 46 | root container of the application, stored in the root of the database |
|---|
| 47 | itself. |
|---|
| 48 | |
|---|
| 49 | This kind of storage requires that each object gets a unique object |
|---|
| 50 | identifier (object id) within its container. The id string is visible |
|---|
| 51 | in the browser address bar. Though it's technically possible for ids |
|---|
| 52 | to contain spaces or slashes we do not allow these kinds of special |
|---|
| 53 | characters in object ids to facilitate the readability of URLs. |
|---|
| 54 | |
|---|
| 55 | Batch Processing |
|---|
| 56 | ================ |
|---|
| 57 | |
|---|
| 58 | Administrators of web portals, which store their data in relational |
|---|
| 59 | databases, are used to getting direct access to the portal's database. |
|---|
| 60 | There are even tools to handle the administration of these databases |
|---|
| 61 | over the Internet, like phpMyAdmin or phpPgAdmin to handle MySQL or |
|---|
| 62 | PostgreSQL databases respectively. These user interfaces bypass the |
|---|
| 63 | portals' user interfaces and give direct access to the database. They |
|---|
| 64 | allow to easily import or export (dump) data tables or the entire |
|---|
| 65 | database structure into CSV or SQL files. What at first sight appears |
|---|
| 66 | to be very helpful and administration-friendly proves to be very |
|---|
| 67 | dangerous on closer inspection. Data structures can be easily damaged |
|---|
| 68 | or destroyed, or data can be easily manipulated by circumventing the |
|---|
| 69 | portal's security machinery or logging system. Kofa does not provide |
|---|
| 70 | any external user interface to access the ZODB_ directly, neither for |
|---|
| 71 | viewing nor for editing data. This includes also the export and import |
|---|
| 72 | of sets of data. Exports and imports are handled via the Kofa user |
|---|
| 73 | interface itself. This is called batch processing which means either |
|---|
| 74 | producing CSV files (comma-separated values) from portal data (export) |
|---|
| 75 | or processing CSV files in order to add, update or remove portal data |
|---|
| 76 | (import). Main premise of Kofa's batch processing technology is that |
|---|
| 77 | the data stored in the ZODB_ can be specifically backed up and |
|---|
| 78 | restored by exporting and importing data. But that's not all. Batch |
|---|
| 79 | processors can do much more. They are an integral part of the student |
|---|
| 80 | registration management. |
|---|
| 81 | |
|---|
| 82 | .. note:: |
|---|
| 83 | |
|---|
| 84 | Although exporters are part of Kofa's batch processing module, we will |
|---|
| 85 | not call them batch processors. Only importers are called batch |
|---|
| 86 | processors. Exporters produce CSV files, importer process them. |
|---|
| 87 | |
|---|
| 88 | |
|---|
| 89 | .. _ZODB: http://www.zodb.org/ |
|---|
