| 1 | .. _testing: |
|---|
| 2 | |
|---|
| 3 | Testing :sup:`in progress` |
|---|
| 4 | ************************** |
|---|
| 5 | |
|---|
| 6 | Introduction |
|---|
| 7 | ============ |
|---|
| 8 | |
|---|
| 9 | Kofa's Python code is being tested automatically. The developers' |
|---|
| 10 | goal is to reach 100% code coverage by Kofa's test runners, which |
|---|
| 11 | means that each single line of code is passed at least one time when |
|---|
| 12 | running the tests. |
|---|
| 13 | |
|---|
| 14 | Why testing? Testing makes sure that our code works properly under |
|---|
| 15 | given sets of conditions and, only comprehensive testing ensures |
|---|
| 16 | that changing or customizing the code does not break existing |
|---|
| 17 | functionality. Why *automated* testing? Simply because no developer |
|---|
| 18 | likes to click around the user interface to check tons of |
|---|
| 19 | functionality. In Kofa more than 1300 tests, with dozens of actions |
|---|
| 20 | per test, are being executed each time when the testrunner is |
|---|
| 21 | started. This job can't be done manually. |
|---|
| 22 | |
|---|
| 23 | Why *automated* testing? Simply because no developer likes to click |
|---|
| 24 | around the user interface to check tons of functionality. In Kofa more |
|---|
| 25 | than 1100 tests, with dozens of actions per test, are being executed |
|---|
| 26 | each time when the testrunner is started. This job can't be done |
|---|
| 27 | manually. |
|---|
| 28 | |
|---|
| 29 | What we test: "unit" and "functional" Tests |
|---|
| 30 | ------------------------------------------- |
|---|
| 31 | |
|---|
| 32 | There are two different ways to test the code of a web application |
|---|
| 33 | automatically: *unit tests* and *functional tests*. The goal of unit |
|---|
| 34 | testing is to isolate each part of the program and show that the |
|---|
| 35 | individual parts are correct. Functional tests of a web application |
|---|
| 36 | are more wholistic, they require a running web application in the |
|---|
| 37 | background with even a working (temporary) database. Functional |
|---|
| 38 | tests are typically linked to a particular use case, they are |
|---|
| 39 | interacting with the portal as a user would. That implies that |
|---|
| 40 | functional testing has to make use of a test browser. A test browser |
|---|
| 41 | does the same job as normal web browser does, except for visualizing |
|---|
| 42 | the rendered HTML code. |
|---|
| 43 | |
|---|
| 44 | |
|---|
| 45 | How we test: "Python" and "Doctest" Tests |
|---|
| 46 | ----------------------------------------- |
|---|
| 47 | |
|---|
| 48 | There are also two different ways to integrate tests, either |
|---|
| 49 | functional or unit, into a Python package: *doc tests* (or doctests) |
|---|
| 50 | and *Python tests*. Python test modules are a collection of |
|---|
| 51 | isolatable Python test cases. A test case combines a collection of |
|---|
| 52 | unit test methods which are being executed by the testrunner one |
|---|
| 53 | after the other. Python test modules are automatically identified by |
|---|
| 54 | means of their filenames which start with ``test_``. In contrast, |
|---|
| 55 | doctests can be wrapped up in simple text files (ending with |
|---|
| 56 | ``.txt``), or they can be put into docstrings of the application's |
|---|
| 57 | source code itself. Common to all doctests is that they are based on |
|---|
| 58 | output from the standard Python interpreter shell (indicated by the |
|---|
| 59 | interpreter prompt ``>>>``. The doctest runner parses all ``py`` and |
|---|
| 60 | ``txt`` files in our package, executes the interpreter commands |
|---|
| 61 | found and compares the output against an expected value. Example:: |
|---|
| 62 | |
|---|
| 63 | Python's `math` module can compute the square root of 2: |
|---|
| 64 | |
|---|
| 65 | >>> from math import sqrt |
|---|
| 66 | >>> sqrt(2) |
|---|
| 67 | 1.4142135623730951 |
|---|
| 68 | |
|---|
| 69 | Why wrapping tests into documentation? An objective of testdriven |
|---|
| 70 | software development is also the documentation of the 'Application |
|---|
| 71 | Programming Interface' (API) and, to a certain extent, providing a |
|---|
| 72 | guideline to users, how to operate the portal. The first is mainly |
|---|
| 73 | done in the docstrings of Python modules which present an expressive |
|---|
| 74 | documentation of the main use cases of a module and its components. |
|---|
| 75 | The latter is primarily done in separate ``txt`` files. |
|---|
| 76 | |
|---|
| 77 | When starting the development of Kofa, we relied on writing a |
|---|
| 78 | coherent documentation including doctests in restructured text |
|---|
| 79 | format. During the software development process the focus shifted |
|---|
| 80 | from doctesting to pure Python testing with a lot of functional |
|---|
| 81 | tests inside. It turned out that Python tests are easier to handle |
|---|
| 82 | and more powerful. Drawback is, that these tests cannot easily be |
|---|
| 83 | integrated into the Sphinx documentation project (the documentation |
|---|
| 84 | which you are reading right now). However, we will list some of |
|---|
| 85 | these tests and try to explain what they are good for. |
|---|
| 86 | |
|---|
| 87 | |
|---|
| 88 | Doctests |
|---|
| 89 | ======== |
|---|
| 90 | |
|---|
| 91 | Browsing |
|---|
| 92 | -------- |
|---|
| 93 | |
|---|
| 94 | .. toctree:: |
|---|
| 95 | :maxdepth: 2 |
|---|
| 96 | |
|---|
| 97 | testing/browser |
|---|
| 98 | testing/breadcrumbs |
|---|
| 99 | |
|---|
| 100 | Cataloging |
|---|
| 101 | ---------- |
|---|
| 102 | |
|---|
| 103 | .. toctree:: |
|---|
| 104 | :maxdepth: 2 |
|---|
| 105 | |
|---|
| 106 | testing/catalog |
|---|
| 107 | |
|---|
| 108 | Data Center |
|---|
| 109 | ----------- |
|---|
| 110 | |
|---|
| 111 | .. toctree:: |
|---|
| 112 | :maxdepth: 2 |
|---|
| 113 | |
|---|
| 114 | testing/datacenter |
|---|
| 115 | |
|---|
| 116 | Security |
|---|
| 117 | -------- |
|---|
| 118 | |
|---|
| 119 | .. toctree:: |
|---|
| 120 | :maxdepth: 2 |
|---|
| 121 | |
|---|
| 122 | testing/permissions |
|---|
| 123 | |
|---|
| 124 | Officers |
|---|
| 125 | -------- |
|---|
| 126 | |
|---|
| 127 | .. toctree:: |
|---|
| 128 | :maxdepth: 2 |
|---|
| 129 | |
|---|
| 130 | testing/userscontainer |
|---|
| 131 | |
|---|
| 132 | Academic Section |
|---|
| 133 | ---------------- |
|---|
| 134 | |
|---|
| 135 | .. toctree:: |
|---|
| 136 | :maxdepth: 2 |
|---|
| 137 | |
|---|
| 138 | testing/certificate |
|---|
| 139 | |
|---|
| 140 | Batch Processing |
|---|
| 141 | ---------------- |
|---|
| 142 | |
|---|
| 143 | .. toctree:: |
|---|
| 144 | :maxdepth: 2 |
|---|
| 145 | |
|---|
| 146 | testing/batching |
|---|
| 147 | testing/batchprocessing |
|---|
| 148 | |
|---|
| 149 | Access Codes |
|---|
| 150 | ------------ |
|---|
| 151 | |
|---|
| 152 | .. toctree:: |
|---|
| 153 | :maxdepth: 2 |
|---|
| 154 | |
|---|
| 155 | testing/accesscode |
|---|
| 156 | |
|---|
| 157 | |
|---|
| 158 | Python Tests |
|---|
| 159 | ============ |
|---|
| 160 | |
|---|
| 161 | .. literalinclude:: ../../../src/waeup/kofa/browser/tests/test_browser.py |
|---|
| 162 | :pyobject: test_suspended_officer |
|---|
