| 1 | .. _testing: |
|---|
| 2 | |
|---|
| 3 | Testing :sup:`in progress` |
|---|
| 4 | ************************** |
|---|
| 5 | |
|---|
| 6 | Introduction |
|---|
| 7 | ============ |
|---|
| 8 | |
|---|
| 9 | Kofa's Python code is being tested automatically. The developers' |
|---|
| 10 | goal is to reach 100% code coverage by Kofa's test runners, which |
|---|
| 11 | means that each single line of code is passed at least one time when |
|---|
| 12 | running the tests. |
|---|
| 13 | |
|---|
| 14 | Why testing? Testing makes sure that our code works properly under |
|---|
| 15 | given sets of conditions and, only comprehensive testing ensures |
|---|
| 16 | that changing or customizing the code does not break existing |
|---|
| 17 | functionality. Why *automated* testing? Simply because no developer |
|---|
| 18 | likes to click around the user interface to check tons of |
|---|
| 19 | functionality. In Kofa more than 1300 tests, with dozens of actions |
|---|
| 20 | per test, are being executed each time when the testrunner is |
|---|
| 21 | started. This job can't be done manually. |
|---|
| 22 | |
|---|
| 23 | There are two different ways to test the code of a web application |
|---|
| 24 | automatically: *unit tests* and *functional tests*. The goal of unit |
|---|
| 25 | testing is to isolate each part of the program and show that the |
|---|
| 26 | individual parts are correct. Functional tests of a web application |
|---|
| 27 | are more wholistic, they require a running web application in the |
|---|
| 28 | background with even a working (temporary) database. Functional |
|---|
| 29 | tests are typically linked to a particular use case, they are |
|---|
| 30 | interacting with the portal as a user would. That implies that |
|---|
| 31 | functional testing has to make use of a test browser. A test browser |
|---|
| 32 | does the same job as normal web browser does, except for visualizing |
|---|
| 33 | the rendered html code. |
|---|
| 34 | |
|---|
| 35 | There are also two different ways to integrate tests, either |
|---|
| 36 | functional or unit, into a Python package: *doc tests* (or doctests) |
|---|
| 37 | and *Python tests*. Python test modules are a collection of |
|---|
| 38 | isolatable Python test cases. A test case combines a collection of |
|---|
| 39 | unit test methods which are being executed by the testrunner one |
|---|
| 40 | after the other. Python test modules are automatically identified by |
|---|
| 41 | means of their filenames which start with ``test_``. In contrast, |
|---|
| 42 | doctests can be wrapped up in simple text files (ending with |
|---|
| 43 | ``.txt``), or they can be put into docstrings of the application's |
|---|
| 44 | source code itself. Common to all doctests is that they are based on |
|---|
| 45 | output from the standard Python interpreter shell (indicated by the |
|---|
| 46 | interpreter prompt ``>>>``. The doctest runner parses all ``py`` and |
|---|
| 47 | ``txt`` files in our package, executes the interpreter commands |
|---|
| 48 | found and compares the output against an expected value. Example:: |
|---|
| 49 | |
|---|
| 50 | Python's `math` module can compute the square root of 2: |
|---|
| 51 | |
|---|
| 52 | >>> from math import sqrt |
|---|
| 53 | >>> sqrt(2) |
|---|
| 54 | 1.4142135623730951 |
|---|
| 55 | |
|---|
| 56 | Why wrapping tests into documentation? An objective of testdriven |
|---|
| 57 | software development is also the documentation of the 'Application |
|---|
| 58 | Programming Interface' (API) and, to a certain extent, providing a |
|---|
| 59 | guideline to users, how to operate the portal. The first is mainly |
|---|
| 60 | done in the docstrings of Python modules which present an expressive |
|---|
| 61 | documentation of the main use cases of a module and its components. |
|---|
| 62 | The latter is primarily done in separate ``txt`` files. |
|---|
| 63 | |
|---|
| 64 | When starting the development of Kofa, we relied on writing a |
|---|
| 65 | coherent documentation including doctests in restructured text |
|---|
| 66 | format. During the software development process the focus shifted |
|---|
| 67 | from doctesting to pure Python testing with a lot of functional |
|---|
| 68 | tests inside. It turned out that Python tests are easier to handle |
|---|
| 69 | and more powerful. Drawback is, that these tests cannot easily be |
|---|
| 70 | integrated into the Sphinx documentation project (the documentation |
|---|
| 71 | which you are reading right now). However, we will list some of |
|---|
| 72 | these tests and try to explain what they are good for. |
|---|
| 73 | |
|---|
| 74 | |
|---|
| 75 | Doctests |
|---|
| 76 | ======== |
|---|
| 77 | |
|---|
| 78 | Browsing |
|---|
| 79 | -------- |
|---|
| 80 | |
|---|
| 81 | .. toctree:: |
|---|
| 82 | :maxdepth: 2 |
|---|
| 83 | |
|---|
| 84 | testing/browser |
|---|
| 85 | testing/breadcrumbs |
|---|
| 86 | |
|---|
| 87 | Cataloging |
|---|
| 88 | ---------- |
|---|
| 89 | |
|---|
| 90 | .. toctree:: |
|---|
| 91 | :maxdepth: 2 |
|---|
| 92 | |
|---|
| 93 | testing/catalog |
|---|
| 94 | |
|---|
| 95 | Data Center |
|---|
| 96 | ----------- |
|---|
| 97 | |
|---|
| 98 | .. toctree:: |
|---|
| 99 | :maxdepth: 2 |
|---|
| 100 | |
|---|
| 101 | testing/datacenter |
|---|
| 102 | |
|---|
| 103 | Security |
|---|
| 104 | -------- |
|---|
| 105 | |
|---|
| 106 | .. toctree:: |
|---|
| 107 | :maxdepth: 2 |
|---|
| 108 | |
|---|
| 109 | testing/permissions |
|---|
| 110 | |
|---|
| 111 | Officers |
|---|
| 112 | -------- |
|---|
| 113 | |
|---|
| 114 | .. toctree:: |
|---|
| 115 | :maxdepth: 2 |
|---|
| 116 | |
|---|
| 117 | testing/userscontainer |
|---|
| 118 | |
|---|
| 119 | Academic Section |
|---|
| 120 | ---------------- |
|---|
| 121 | |
|---|
| 122 | .. toctree:: |
|---|
| 123 | :maxdepth: 2 |
|---|
| 124 | |
|---|
| 125 | testing/certificate |
|---|
| 126 | |
|---|
| 127 | Batch Processing |
|---|
| 128 | ---------------- |
|---|
| 129 | |
|---|
| 130 | .. toctree:: |
|---|
| 131 | :maxdepth: 2 |
|---|
| 132 | |
|---|
| 133 | testing/batching |
|---|
| 134 | testing/batchprocessing |
|---|
| 135 | |
|---|
| 136 | Access Codes |
|---|
| 137 | ------------ |
|---|
| 138 | |
|---|
| 139 | .. toctree:: |
|---|
| 140 | :maxdepth: 2 |
|---|
| 141 | |
|---|
| 142 | testing/accesscode |
|---|
| 143 | |
|---|
| 144 | |
|---|
| 145 | Pythontests |
|---|
| 146 | =========== |
|---|
