source: main/waeup.kofa/trunk/docs/source/userdocs/datacenter/export.rst @ 15128

Last change on this file since 15128 was 15051, checked in by Henrik Bettermann, 6 years ago
File size: 14.1 KB
RevLine 
[12863]1.. _export:
2
3Data Export
4***********
5
[12870]6Regular data exporters (1) collect objects from specific containers,
7(2) iterate over the collected objects, (3) extract and mangle
8information from each object, (4) write the information of each object
9into a row of a CSV file and (5) finally provide the file for
10download. The CSV file is neither stored in the database nor archived
11in the filesystem. (3) and (4) means a flattening of the hierarchical
12data structure, i.e. a mapping of objects to flat relational data to
13be stored in a CSV table. The extracted information must not
14necessarily be based only on static attributes of the collected
15object. The data, finally stored in the CSV file, can also be derived
16from parent or child objects, or dynamically computed by the object's
17methods and property attributes. These methods and properties can
18retrieve information from everywhere in the portal's database. In the
19following we list all exporter classes including two attributes and a
20method description. The `fields` attribute contain the column titles
21of the export file. These are not necessarily only attributes of the
22exported objects.
[12863]23
24.. note::
25
[12883]26  The list of exported columns usually underlies heavy customizations.
27  In the Kofa base package only very few columns are being exported. In
28  some Kofa custom packages tons of data are being gathered from
[13104]29  applicants and students and the number of columns increases
30  accordingly.
[12863]31
[12870]32The `title` attribute unveils the name of the exporter under which
33this exporter will be displayed in the user interface. The
34`mangle_value()` method shows how some of fields are being dynamically
35computed.
[12863]36
37Regular Exporters
38=================
39
40User Exporter
41-------------
42
43.. autoclass:: waeup.kofa.userscontainer.UserExporter()
44
45  .. autoattribute:: waeup.kofa.userscontainer.UserExporter.fields
46  .. autoattribute:: waeup.kofa.userscontainer.UserExporter.title
47  .. automethod:: waeup.kofa.userscontainer.UserExporter.mangle_value()
48
49Faculty Exporter
50----------------
51
52.. autoclass:: waeup.kofa.university.export.FacultyExporter()
53
54  .. autoattribute:: waeup.kofa.university.export.FacultyExporter.fields
55  .. autoattribute:: waeup.kofa.university.export.FacultyExporter.title
56  .. automethod:: waeup.kofa.university.export.FacultyExporter.mangle_value()
57
58Department Exporter
59-------------------
60
61.. autoclass:: waeup.kofa.university.export.DepartmentExporter()
62
63  .. autoattribute:: waeup.kofa.university.export.DepartmentExporter.fields
64  .. autoattribute:: waeup.kofa.university.export.DepartmentExporter.title
65  .. automethod:: waeup.kofa.university.export.DepartmentExporter.mangle_value()
66
67Course Exporter
68---------------
69
70.. autoclass:: waeup.kofa.university.export.CourseExporter()
71
72  .. autoattribute:: waeup.kofa.university.export.CourseExporter.fields
73  .. autoattribute:: waeup.kofa.university.export.CourseExporter.title
74  .. automethod:: waeup.kofa.university.export.CourseExporter.mangle_value()
75
76Certificate Exporter
77--------------------
78
79.. autoclass:: waeup.kofa.university.export.CertificateExporter()
80
81  .. autoattribute:: waeup.kofa.university.export.CertificateExporter.fields
82  .. autoattribute:: waeup.kofa.university.export.CertificateExporter.title
83  .. automethod:: waeup.kofa.university.export.CertificateExporter.mangle_value()
84
85Certificate Course Exporter
86---------------------------
87
88.. autoclass:: waeup.kofa.university.export.CertificateCourseExporter()
89
90  .. autoattribute:: waeup.kofa.university.export.CertificateCourseExporter.fields
91  .. autoattribute:: waeup.kofa.university.export.CertificateCourseExporter.title
92  .. automethod:: waeup.kofa.university.export.CertificateCourseExporter.mangle_value()
93
94Access Code Batch Exporter
95--------------------------
96
97.. autoclass:: waeup.kofa.accesscodes.export.AccessCodeBatchExporter()
98
99  .. autoattribute:: waeup.kofa.accesscodes.export.AccessCodeBatchExporter.fields
100  .. autoattribute:: waeup.kofa.accesscodes.export.AccessCodeBatchExporter.title
101
102Access Code Exporter
103--------------------
104
105.. autoclass:: waeup.kofa.accesscodes.export.AccessCodeExporter()
106
107  .. autoattribute:: waeup.kofa.accesscodes.export.AccessCodeExporter.fields
108  .. autoattribute:: waeup.kofa.accesscodes.export.AccessCodeExporter.title
109  .. automethod:: waeup.kofa.accesscodes.export.AccessCodeExporter.mangle_value()
110
111Hostel Exporter
112---------------
113
114.. autoclass:: waeup.kofa.hostels.export.HostelExporter()
115
116  .. autoattribute:: waeup.kofa.hostels.export.HostelExporter.fields
117  .. autoattribute:: waeup.kofa.hostels.export.HostelExporter.title
118
119Bed Exporter
120------------
121
122.. autoclass:: waeup.kofa.hostels.export.BedExporter()
123
124  .. autoattribute:: waeup.kofa.hostels.export.BedExporter.fields
125  .. autoattribute:: waeup.kofa.hostels.export.BedExporter.title
126
[13144]127Document Exporter
128-----------------
129
130.. autoclass:: waeup.kofa.documents.export.DocumentExporterBase()
131
132  .. automethod:: waeup.kofa.documents.export.DocumentExporterBase.mangle_value()
133
[12863]134Application Data Exporters
135==========================
136
137Applicants Container Exporter
138-----------------------------
139
140.. autoclass:: waeup.kofa.applicants.export.ApplicantsContainerExporter()
141
142  .. autoattribute:: waeup.kofa.applicants.export.ApplicantsContainerExporter.fields
143  .. autoattribute:: waeup.kofa.applicants.export.ApplicantsContainerExporter.title
144
145Applicant Exporter
146------------------
147
148.. autoclass:: waeup.kofa.applicants.export.ApplicantExporter()
149
150  .. autoattribute:: waeup.kofa.applicants.export.ApplicantExporter.fields
151  .. autoattribute:: waeup.kofa.applicants.export.ApplicantExporter.title
152  .. automethod:: waeup.kofa.applicants.export.ApplicantExporter.mangle_value()
153
[13949]154Applicant Payment Exporter
155--------------------------
156
157.. autoclass:: waeup.kofa.applicants.export.ApplicantPaymentExporter()
158
159  .. autoattribute:: waeup.kofa.applicants.export.ApplicantPaymentExporter.fields
160  .. autoattribute:: waeup.kofa.applicants.export.ApplicantPaymentExporter.title
161  .. automethod:: waeup.kofa.applicants.export.ApplicantPaymentExporter.mangle_value()
162
[12863]163Student Data Exporters
[12865]164======================
[12863]165
[12870]166When starting a Student Data Exporter in the Data Center all student
167records will be taken into consideration, no matter what or where a
168student is studying. The exporter can also be started 'locally' at
169various levels in the academic section. Starting one of the exporters
170e.g. at faculty or department level means that only the data of
171students are exported who study in this faculty or department
172respectively. The exporter can also be started at certificate level.
173Then only the data of students, who are studying the named study
174course, will be taken into account. At course level the data of those
175students are being exported who have attended or taken this specific
176course.
[12863]177
[12870]178Student Data Exporter can be further configured through a
179configuration page. Search parameters like the student's current level,
[12883]180current session and current study mode can be set to filter sets of
[12870]181students in order to decrease the size of the export file. The set of
182filter parameters varies and depends on the 'locatation' from where
183the exporter is called. A completely different set of filter
184parameters is provided for courses. In this case the session and level
185can be selected when the course was taken by the student.
[12863]186
187Student Exporter
188----------------
189
190.. autoclass:: waeup.kofa.students.export.StudentExporter()
191
192  .. autoattribute:: waeup.kofa.students.export.StudentExporter.fields
193  .. autoattribute:: waeup.kofa.students.export.StudentExporter.title
194  .. automethod:: waeup.kofa.students.export.StudentExporter.mangle_value()
195
196Student Study Course Exporter
197-----------------------------
198
199.. autoclass:: waeup.kofa.students.export.StudentStudyCourseExporter()
200
201  .. autoattribute:: waeup.kofa.students.export.StudentStudyCourseExporter.fields
202  .. autoattribute:: waeup.kofa.students.export.StudentStudyCourseExporter.title
203  .. automethod:: waeup.kofa.students.export.StudentStudyCourseExporter.mangle_value()
204
205Student Study Level Exporter
206----------------------------
207
208.. autoclass:: waeup.kofa.students.export.StudentStudyLevelExporter()
209
210  .. autoattribute:: waeup.kofa.students.export.StudentStudyLevelExporter.fields
211  .. autoattribute:: waeup.kofa.students.export.StudentStudyLevelExporter.title
212  .. automethod:: waeup.kofa.students.export.StudentStudyLevelExporter.mangle_value()
213
214Course Ticket Exporter
215----------------------
216
217.. autoclass:: waeup.kofa.students.export.CourseTicketExporter()
218
219  .. autoattribute:: waeup.kofa.students.export.CourseTicketExporter.fields
220  .. autoattribute:: waeup.kofa.students.export.CourseTicketExporter.title
221  .. automethod:: waeup.kofa.students.export.CourseTicketExporter.mangle_value()
222
[12865]223Student Payment Exporter
224------------------------
[12863]225
[12865]226.. autoclass:: waeup.kofa.students.export.StudentPaymentExporter()
[12863]227
[12865]228  .. autoattribute:: waeup.kofa.students.export.StudentPaymentExporter.fields
229  .. autoattribute:: waeup.kofa.students.export.StudentPaymentExporter.title
230  .. automethod:: waeup.kofa.students.export.StudentPaymentExporter.mangle_value()
[12863]231
[12971]232Student Unpaid Payment Exporter
233-------------------------------
234
235.. autoclass:: waeup.kofa.students.export.StudentUnpaidPaymentExporter()
236
237  .. autoattribute:: waeup.kofa.students.export.StudentUnpaidPaymentExporter.title
238
[12865]239Bed Ticket Exporter
240-------------------
[12863]241
[12865]242.. autoclass:: waeup.kofa.students.export.BedTicketExporter()
[12863]243
[12865]244  .. autoattribute:: waeup.kofa.students.export.BedTicketExporter.fields
245  .. autoattribute:: waeup.kofa.students.export.BedTicketExporter.title
246  .. automethod:: waeup.kofa.students.export.BedTicketExporter.mangle_value()
[12863]247
[12884]248.. note::
[12863]249
[12884]250  The above exporters refer to a specific content type (object class).
251  They export all attributes of these objects and a few additional
252  parameters derived from the parent objects. These exporters can be
253  used for reimport, or more precisely for backing up and restoring
254  data. The following 'special' exporters are made on request of some
255  universities to collect and compose student data for analysis and
256  postprocessing by the university.
257
[15047]258Data For Bursary Exporter
259-------------------------
[12863]260
261.. autoclass:: waeup.kofa.students.export.DataForBursaryExporter()
262
263  .. autoattribute:: waeup.kofa.students.export.DataForBursaryExporter.fields
264  .. autoattribute:: waeup.kofa.students.export.DataForBursaryExporter.title
265  .. automethod:: waeup.kofa.students.export.DataForBursaryExporter.mangle_value()
266
[15047]267School Fee Payments Overview Exporter
268-------------------------------------
[12863]269
[15047]270.. autoclass:: waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter()
[12863]271
[15047]272  .. autoattribute:: waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter.fields
273  .. autoattribute:: waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter.title
274  .. autoattribute:: waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter.curr_year
275  .. automethod:: waeup.kofa.students.export.SchoolFeePaymentsOverviewExporter.mangle_value()
[12863]276
[15051]277Session Payments Overview Exporter
278----------------------------------
279
280.. autoclass:: waeup.kofa.students.export.SessionPaymentsOverviewExporter()
281
282  .. autoattribute:: waeup.kofa.students.export.SessionPaymentsOverviewExporter.fields
283  .. autoattribute:: waeup.kofa.students.export.SessionPaymentsOverviewExporter.title
284  .. autoattribute:: waeup.kofa.students.export.SessionPaymentsOverviewExporter.curr_year
285  .. automethod:: waeup.kofa.students.export.SessionPaymentsOverviewExporter.mangle_value()
286
[12863]287Student Study Levels Overview Exporter
288--------------------------------------
289
290.. autoclass:: waeup.kofa.students.export.StudentStudyLevelsOverviewExporter()
291
292  .. autoattribute:: waeup.kofa.students.export.StudentStudyLevelsOverviewExporter.fields
293  .. autoattribute:: waeup.kofa.students.export.StudentStudyLevelsOverviewExporter.title
294  .. automethod:: waeup.kofa.students.export.StudentStudyLevelsOverviewExporter.mangle_value()
295
296Combo Card Data Exporter
297------------------------
298
299.. autoclass:: waeup.kofa.students.export.ComboCardDataExporter()
300
301  .. autoattribute:: waeup.kofa.students.export.ComboCardDataExporter.fields
302  .. autoattribute:: waeup.kofa.students.export.ComboCardDataExporter.title
[13162]303  .. automethod:: waeup.kofa.students.export.ComboCardDataExporter.mangle_value()
304
[15047]305Data For Lecturer Exporter
306--------------------------
307
308.. autoclass:: waeup.kofa.students.export.DataForLecturerExporter()
309
310  .. autoattribute:: waeup.kofa.students.export.DataForLecturerExporter.fields
311  .. autoattribute:: waeup.kofa.students.export.DataForLecturerExporter.title
312  .. automethod:: waeup.kofa.students.export.DataForLecturerExporter.mangle_value()
313
[13162]314File Export
315===========
316
317You want to export files (passport images, pdf slips), for instance
318of all students in a department or applicants in an applicants
319container? There is no facility in Kofa which does this job for you,
320but you can batch-download files by means of your operating system
321on your local machine. This works perfectly with the the `wget`
322command on Linux or MacOS computers. Windows does not provide such a
323command.
324
325If ``numbers.txt`` contains the application numbers of applicants in
326the applicants container ``xyz``, the following bash script will
327download all passport images directly onto you computer::
328
329  wget --save-cookies cookies.txt --keep-session-cookies --post-data 'form.login=my-username&form.password=my-password' https://my-kofa-site/login
330
331  for i in $(cat numbers.txt)
332  do
[13189]333   wget --load-cookies cookies.txt --output-document=$i.jpg https://my-kofa-site/applicants/xyz/$i/passport.jpg
[13162]334  done
335
336If ``numbers.txt`` contains the ids of students in a department, the
337following bash script will download all passport images directly
338onto you computer. The script is a slightly extended, more user
339friendly version of the script above::
340
341  if [ $# -lt 3 ]
342  then
343     echo "usage: $0 username password filename"
344     exit 1
345  fi
346
347  wget --save-cookies cookies.txt --keep-session-cookies --post-data "form.login=$1&form.password=$2" https://my-kofa-site/login
348
[14780]349  counter=0
350
[13162]351  while read variable
352  do
[14780]353     counter=$((counter+1))
[13162]354     wget --load-cookies cookies.txt --output-document $variable.jpg https://my-kofa-site/students/$variable/passport.jpg
[14780]355     if [[ "$counter" -gt 1000 ]]; then
356        wget --save-cookies cookies.txt --keep-session-cookies --post-data "form.login=$1&form.password=$2" https://my-kofa-site/login
357        counter=0
358     fi
[13162]359  done < $3
Note: See TracBrowser for help on using the repository browser.