The test suite depends on
pip install tox pytest
To run the tests for all supported Python versions:
If you want to test only a specific Python version (e.g. Python 2.7), it’s as easy as:
tox -e py27
Tests are also run automatically on Travis CI.
You want to add a test for Jedi? Great! We love that. Normally you should write your tests as Blackbox Tests. Most tests would fit right in there.
For specific API testing we’re using simple unit tests, with a focus on a simple and readable testing structure.
Blackbox Tests (run.py)¶
Jedi is mostly being tested by what I would call “Blackbox Tests”. These
tests are just testing the interface and do input/output testing. This makes a
lot of sense for Jedi. Jedi supports so many different code structures, that
it is just stupid to write 200‘000 unittests in the manner of
regression.py. Also, it is impossible to do doctests/unittests on most of
the internal data structures. That’s why Jedi uses mostly these kind of
There are different kind of tests:
- completions / goto_definitions
How to run tests?¶
Integration test cases are located in
and each test case is indicated by either the comment
#? (completions /
#! (assignments), or
There is also support for third party libraries. In a normal test run they are
not being executed, you have to provide a
In addition to standard -k and -m options in py.test, you can use
-T (–test-files) option to specify integration test cases to run.
It takes the format of
FILE_NAME is a file in
LINE is a line
number of the test comment. Here is some recipes:
Run tests only in
py.test test/test_integration.py -T basic.py -T imports.py
Run test at line 4, 6, and 8 in
py.test test/test_integration.py -T basic.py:4,6,8
py.test --help for more information.
If you want to debug a test, just use the
Alternate Test Runner¶
If you don’t like the output of
py.test, there’s an alternate test runner
that you can start by running
./run.py. The above example could be run by:
./run.py basic 4 6 8 50-80
The advantage of this runner is simplicity and more customized error reports. Using both runners will help you to have a quicker overview of what’s happening.
Uses comments to specify a test in the next line. The comment says which results are expected. The comment always begins with #?. The last row symbolizes the cursor.
#? ['real'] a = 3; a.rea
Because it follows
a.rea and a is an
int, which has a
Definition tests use the same symbols like completion tests. This is possible because the completion tests are defined with a list:
#? int() ab = 3; ab
Tests look like this:
abc = 1 #! ['abc=1'] abc
Additionally it is possible to specify the column by adding a number, which describes the position of the test (otherwise it’s just the end of line):
#! 2 ['abc=1'] abc
Tests look like this:
abc = 1 #< abc@1,0 abc@3,0 abc
Refactoring Tests (refactor.py)¶
Refactoring tests work a little bit similar to Black Box tests. But the idea is here to compare two versions of code. Note: Refactoring is currently not in active development (and was never stable), the tests are therefore not really valuable - just ignore them.