Python imports for testing modules with additional dependencies

Multi tool use
Python imports for testing modules with additional dependencies
I want to have my tests in a separate folder from my package code, such that from the top level directory of my project I can run python sample/run.py
or python tests/test_run.py
, and have both of them resolve all the imports properly.
python sample/run.py
python tests/test_run.py
My directory structure looks like this:
sample/
__init__.py
helper.py
run.py
tests/
context.py
test_run.py
I know there are supposedly many ways to achieve this, as discussed here: Python imports for tests using nose - what is best practice for imports of modules above current package
However, when I try to run python tests/test_run.py
, I get a ModuleNotFoundError for 'helper', because 'sample/run.py' imports 'sample/helper.py'.
python tests/test_run.py
In particular, I am trying to follow the convention (suggested in the Hitchhiker's Guide to Python) of explicitly modifying the path using:
import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
As a result, I have a blank sample/__init__.py
, along with the following code files.
sample/__init__.py
sample/run.py:
from helper import helper_fn
def run():
helper_fn(5)
return 'foo'
if __name__ == '__main__':
run()
sample/helper.py:
def helper_fn(N):
print(list(range(N)))
tests/context.py:
import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import sample
tests/test_run.py:
from context import sample
from sample import run
assert run.run() == 'foo'
So I have two questions:
sample/run.py
tests/test_run.py
Python 3, thanks for clarifying
– camall3n
Jul 2 at 21:46
1 Answer
1
Edited:
To make both sample/run.py
and tests/test_run.py
work, you should add the path of sample
directory into python path. So, your tests/context.py
should be
sample/run.py
tests/test_run.py
sample
tests/context.py
import os, sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../sample')))
import sample
This change will let Python know the path of helper
module.
helper
sample/run.py
should be:
sample/run.py
from .helper import helper_fn
def run():
helper_fn(5)
return 'foo'
if __name__ == '__main__':
run()
Implicit relative imports within packages are not available in Python 3. Please check below:
The import system has been updated to fully implement the second phase of PEP 302. There is no longer any implicit import machinery - the full import system is exposed through sys.meta_path. In addition, native namespace package support has been implemented (see PEP 420). link
This documentation might be helpful to understand Intra-Package-References.
This fixes the import error when running
tests/test_run.py
, but it leads to ModuleNotFoundError: No module named '__main__.helper'; '__main__' is not a package
when running sample/run.py
.– camall3n
Jul 3 at 3:46
tests/test_run.py
ModuleNotFoundError: No module named '__main__.helper'; '__main__' is not a package
sample/run.py
The linked documentation says it's not possible to use relative imports in a module intended for use as the main module, so I tried abstracting the
if __name__ == 'main': run()
into a new file called sample/main.py
(along with from run import run
). When using the relative import suggested above (from .helper ...
), it works for tests/test_run.py
and fails for sample/main.py
. With the original import (from helper ...
), it works for sample/main.py
but not tests/test_run.py
.– camall3n
Jul 3 at 4:03
if __name__ == 'main': run()
sample/main.py
from run import run
from .helper ...
tests/test_run.py
sample/main.py
from helper ...
sample/main.py
tests/test_run.py
The relative import suggested above does work if you run
python -m sample.run
, as mentioned here: stackoverflow.com/a/23542795. I still find this pretty unsatisfying though, since now there are two ways you need to invoke python, depending on whether it's a test script or a project script. Is there any way where I can still call both scripts as specified at the beginning of the question?– camall3n
Jul 3 at 4:20
python -m sample.run
@camall3n I edited my solution. This change will make both codes work and can answer your first question.
– YoungChoi
Jul 3 at 4:20
This still seems kind of ugly, but I'll admit that it does what I asked for. I'm accepting this answer unless someone can think of a cleaner solution.
– camall3n
Jul 3 at 4:36
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Python 2 or Python 3? The way that imports are looked up is slightly different in both.
– jwodder
Jul 2 at 21:44