1 | .. _testing: |
---|
2 | |
---|
3 | Testing :sup:`in progress` |
---|
4 | ************************** |
---|
5 | |
---|
6 | Introduction |
---|
7 | ============ |
---|
8 | |
---|
9 | Kofa's Python code is being tested automatically. The developers' |
---|
10 | goal is to reach 100% code coverage by Kofa's test runners, which |
---|
11 | means that each single line of code is passed at least one time when |
---|
12 | running the tests. |
---|
13 | |
---|
14 | Why testing? Testing makes sure that our code works properly under |
---|
15 | given sets of conditions and, only comprehensive testing ensures |
---|
16 | that changing or customizing the code does not break existing |
---|
17 | functionality. Why *automated* testing? Simply because no developer |
---|
18 | likes to click around the user interface to check tons of |
---|
19 | functionality. In Kofa more than 1300 tests, with dozens of actions |
---|
20 | per test, are being executed each time when the testrunner is |
---|
21 | started. This job can't be done manually. |
---|
22 | |
---|
23 | There are two different ways to test the code of a web application |
---|
24 | automatically: *unit tests* and *functional tests*. The goal of unit |
---|
25 | testing is to isolate each part of the program and show that the |
---|
26 | individual parts are correct. Functional tests of a web application |
---|
27 | are more wholistic, they require a running web application in the |
---|
28 | background with even a working (temporary) database. Functional |
---|
29 | tests are typically linked to a particular use case, they are |
---|
30 | interacting with the portal as a user would. That implies that |
---|
31 | functional testing has to make use of a test browser. A test browser |
---|
32 | does the same job as normal web browser does, except for visualizing |
---|
33 | the rendered html code. |
---|
34 | |
---|
35 | There are also two different ways to integrate tests, either |
---|
36 | functional or unit, into a Python package: *doc tests* (or doctests) |
---|
37 | and *Python tests*. Python test modules are a collection of |
---|
38 | isolatable Python test cases. A test case combines a collection of |
---|
39 | unit test methods which are being executed by the testrunner one |
---|
40 | after the other. Python test modules are automatically identified by |
---|
41 | means of their filenames which start with ``test_``. In contrast, |
---|
42 | doctests can be wrapped up in simple text files (ending with |
---|
43 | ``.txt``), or they can be put into docstrings of the application's |
---|
44 | source code itself. Common to all doctests is that they are based on |
---|
45 | output from the standard Python interpreter shell (indicated by the |
---|
46 | interpreter prompt ``>>>``. The doctest runner parses all ``py`` and |
---|
47 | ``txt`` files in our package, executes the interpreter commands |
---|
48 | found and compares the output against an expected value. Example:: |
---|
49 | |
---|
50 | Python's `math` module can compute the square root of 2: |
---|
51 | |
---|
52 | >>> from math import sqrt |
---|
53 | >>> sqrt(2) |
---|
54 | 1.4142135623730951 |
---|
55 | |
---|
56 | Why wrapping tests into documentation? An objective of testdriven |
---|
57 | software development is also the documentation of the 'Application |
---|
58 | Programming Interface' (API) and, to a certain extent, providing a |
---|
59 | guideline to users, how to operate the portal. The first is mainly |
---|
60 | done in the docstrings of Python modules which present an expressive |
---|
61 | documentation of the main use cases of a module and its components. |
---|
62 | The latter is primarily done in separate ``txt`` files. |
---|
63 | |
---|
64 | When starting the development of Kofa, we relied on writing a |
---|
65 | coherent documentation including doctests in restructured text |
---|
66 | format. During the software development process the focus shifted |
---|
67 | from doctesting to pure Python testing with a lot of functional |
---|
68 | tests inside. It turned out that Python tests are easier to handle |
---|
69 | and more powerful. Drawback is, that these tests cannot easily be |
---|
70 | integrated into the Sphinx documentation project (the documentation |
---|
71 | which you are reading right now). However, we will list some of |
---|
72 | these tests and try to explain what they are good for. |
---|
73 | |
---|
74 | |
---|
75 | Doctests |
---|
76 | ======== |
---|
77 | |
---|
78 | Browsing |
---|
79 | -------- |
---|
80 | |
---|
81 | .. toctree:: |
---|
82 | :maxdepth: 2 |
---|
83 | |
---|
84 | testing/browser |
---|
85 | testing/breadcrumbs |
---|
86 | |
---|
87 | Cataloging |
---|
88 | ---------- |
---|
89 | |
---|
90 | .. toctree:: |
---|
91 | :maxdepth: 2 |
---|
92 | |
---|
93 | testing/catalog |
---|
94 | |
---|
95 | Data Center |
---|
96 | ----------- |
---|
97 | |
---|
98 | .. toctree:: |
---|
99 | :maxdepth: 2 |
---|
100 | |
---|
101 | testing/datacenter |
---|
102 | |
---|
103 | Security |
---|
104 | -------- |
---|
105 | |
---|
106 | .. toctree:: |
---|
107 | :maxdepth: 2 |
---|
108 | |
---|
109 | testing/permissions |
---|
110 | |
---|
111 | Officers |
---|
112 | -------- |
---|
113 | |
---|
114 | .. toctree:: |
---|
115 | :maxdepth: 2 |
---|
116 | |
---|
117 | testing/userscontainer |
---|
118 | |
---|
119 | Academic Section |
---|
120 | ---------------- |
---|
121 | |
---|
122 | .. toctree:: |
---|
123 | :maxdepth: 2 |
---|
124 | |
---|
125 | testing/certificate |
---|
126 | |
---|
127 | Batch Processing |
---|
128 | ---------------- |
---|
129 | |
---|
130 | .. toctree:: |
---|
131 | :maxdepth: 2 |
---|
132 | |
---|
133 | testing/batching |
---|
134 | testing/batchprocessing |
---|
135 | |
---|
136 | Access Codes |
---|
137 | ------------ |
---|
138 | |
---|
139 | .. toctree:: |
---|
140 | :maxdepth: 2 |
---|
141 | |
---|
142 | testing/accesscode |
---|
143 | |
---|
144 | |
---|
145 | Python Tests |
---|
146 | ============ |
---|
147 | |
---|
148 | .. literalinclude:: ../../../src/waeup/kofa/browser/tests/test_browser.py |
---|
149 | :pyobject: SupplementaryBrowserTests.test_suspended_officer |
---|
150 | |
---|