Working with PyTest and Nose¶
It is easy to integrate Glusto capability into a PyTest or Nose formatted test script. Simply add the import to the script and all of the Glusto methods are available.
import pytest from glusto.core import Glusto as g
The Glusto command-line utility currently wraps the PyUnit mechanism for discovering and running tests. PyUnit tests using Glusto can also run under PyTest and, with some exceptions, Nose.
Note
Except where noted, the examples provided here are currently only written
with the Python unittest
module in mind. This section discusses running
those scripts under PyTest and Nose. At some point, I will write a set of
specific examples for each. For now, this discussion is intended to demonstrate
the flexibility Glusto as a library of utilities can provide across the
three popular test frameworks.
The following examples use the tests included in the tests
directory.
PyTest¶
With the provided examples, PyTest runs the tests without issue and even handles skipped tests and expected failures. There are some notable exceptions they make (e.g., ignoring test order with load_tests).
A simple py.test run:
$ py.test ==================== test session starts ============================= platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 rootdir: /home/loadtheaccumulator/glusto, inifile: collected 21 items test_glusto.py x..s.. test_glusto_configs.py ... test_glusto_pytest.py x..s.. test_glusto_rpyc.py ... test_glusto_templates.py ... ============= 17 passed, 2 skipped, 2 xfailed in 2.63 seconds ========The same run with verbose output:
$ py.test -v ================== test session starts =============================== platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /usr/bin/python cachedir: ../.cache rootdir: /home/loadtheaccumulator/Dropbox/glusto, inifile: collected 21 items test_glusto.py::TestGlustoBasics::test_expected_fail <- ../../../../usr/lib64/python2.7/unittest/case.py xfail test_glusto.py::TestGlustoBasics::test_negative_test PASSED test_glusto.py::TestGlustoBasics::test_return_code PASSED test_glusto.py::TestGlustoBasics::test_skip_me <- ../../../../usr/lib64/python2.7/unittest/case.py SKIPPED test_glusto.py::TestGlustoBasics::test_stderr PASSED test_glusto.py::TestGlustoBasics::test_stdout PASSED test_glusto_configs.py::TestGlustoConfigs::test_ini PASSED test_glusto_configs.py::TestGlustoConfigs::test_ini_ordered PASSED test_glusto_configs.py::TestGlustoConfigs::test_yaml PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_expected_fail <- ../../../../usr/lib64/python2.7/unittest/case.py xfail test_glusto_pytest.py::TestGlustoBasicsPyTest::test_negative_test PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_return_code PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_skip_me SKIPPED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_stderr PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_stdout PASSED test_glusto_rpyc.py::TestGlustoRpyc::test_connection PASSED test_glusto_rpyc.py::TestGlustoRpyc::test_local_module_on_remote PASSED test_glusto_rpyc.py::TestGlustoRpyc::test_remote_call PASSED test_glusto_templates.py::TestGlustoTemplates::test_template_forloop PASSED test_glusto_templates.py::TestGlustoTemplates::test_template_include PASSED test_glusto_templates.py::TestGlustoTemplates::test_template_scalar PASSED ============== 17 passed, 2 skipped, 2 xfailed in 2.85 seconds ========
PyTest supports running PyUnit and Nose tests, so it’s simple to leverage PyTest
features, such as markers, in a PyUnit script. See the test_glusto_pytest.py
script for an example of combining PyTest marker features in a unittest.
A test run of only tests with a py.test formatted marker response:
$ py.test -v -m "response" ===================== test session starts ============================ platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /usr/bin/python cachedir: ../.cache rootdir: /home/loadtheaccumulator/Dropbox/glusto, inifile: collected 21 items test_glusto_pytest.py::TestGlustoBasicsPyTest::test_return_code PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_stderr PASSED test_glusto_pytest.py::TestGlustoBasicsPyTest::test_stdout PASSED ============== 18 tests deselected by "-m 'response'" ================= ============== 3 passed, 18 deselected in 0.69 seconds ================
Nose¶
Nose successfully runs the test methods, but does not handle the load_tests function as easily as PyTest.
$ nosetests /usr/lib64/python2.7/unittest/case.py:378: RuntimeWarning: TestResult has no addExpectedFailure method, reporting as passes RuntimeWarning) ...S..E.........E...... ====================================================================== ERROR: Load tests in a specific order. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) TypeError: load_tests() takes exactly 3 arguments (0 given) ====================================================================== ERROR: Load tests in a specific order. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) TypeError: load_tests() takes exactly 3 arguments (0 given) ---------------------------------------------------------------------- Ran 23 tests in 2.253s FAILED (SKIP=1, errors=2)
Twenty-three tests run successfully, two with errors (expected though), and 1 skipped. Apparently Nose ignores the py.test markers and did not skip a test in the py.test example.
A Nose test run with results written to an xunit xml file:
$ nosetests --with-xunit --xunit-file=/tmp/nosetests.xml
Glusto for Good Measure¶
$ glusto -d 'tests' Starting glusto via main() ... ---------------------------------------------------------------------- Ran 21 tests in 2.522s OK (skipped=1, expected failures=2)
Not surprisingly, the unittest
module does not recognize the PyTest skip marker,
so it is currently necessary to run PyTest-savvy scripts with the py.test
command.
To Do¶
- split this out and merge into individual test framework pages
- Add examples of PyTest and Nose specific test scripts using Glusto calls.
more on this subject later…