Imports in Python

Python imports — invaluable part of the language. Source: Pexels, Christina Morillo

Any program you start developing, which turns into a multi-file project has the same problem. How do you use code from a different file? In Python, this is done with the import keyword. But it isn't as easy as you think.

The problem

Once you get to the complexity which requires not simply using modules but creating an entire package, things get messy. But let’s start with a simple script.

Imports aren’t necessary but they’re inevitable

Take a look at this script:

def run():
   print("This script has no imports")

It does not need anything but standard language features, therefore it has no imports. It’s pretty boring.

Let’s say, however, that it needs to print the message in multiple languages. We can make it process a parameter, which defines the language and then open a file to read the text from there — a standard technique for internationalization. Like this:

import sys
def run():
    print("This script has no imports")
if __name__ == "__main__":
    if len(sys.argv) > 1:
        language_code = sys.argv[1]
    else:
        language_code = "en"
        with open("i18n_" + language_code + ".properties") as f:
        translation = f.readlines()[0] # the only line we need is the first one in the file 
run(translation)

You can see that we already have an import here. So it is just a matter of time to use one. In any case, this isn’t that hard and everything works out of the box. But now we have a pretty complex module with a bunch of different responsibilities in one place.

Bringing more complexity

We know that different responsibilities should not be mixed, so we want to split off reading the file into a separate module. We can move the file operations to a separate file called fileops.py

def read_translation(language_code):
    with open("i18n_" + language_code + ".properties") as f:
        translation = f.readlines()[0] # the only line we need is the first one in the file return     
                                                           #translation

With this action, we do not have access to the file I/O operations in our main file. Therefore we need to import the new module into our main.py:

import fileops

and then access the new function there like that:

translation = fileops.read_translation(language_code)

Introducing packages

By now we have only two modules in a single directory, but we’re getting into a stage where we would need to organize them into an entire package with submodules.

To do this we would have to introduce a __init__.py file in the directory where our modules stay. Then if needed, we can create subdirectories where separate modules would be located. Each subdirectory will have its own __init__.py file. Our package would look something like this:

awesomepackage 
  |- __init__.py
  |- main.py
  |-io
      |-__init__.py 
      |-fileops.py

And here start the problems

Just like that, the project is already complex enough by itself to start messing it up. On top of that, if you are just starting with Python, it starts fighting you back.

The symptoms

If we now try and run our simple application it will, naturally, fail. This is because we moved the fileops module to another submodule. So we need to change the import statement:

import io.fileops

But if we run our program now, there is an error message:

ModuleNotFoundError: No module named 'io.fileops'; 'io' is not a package

So this doesn’t look good. But Python offers more than one way to import a module. Perhaps there is a difference in the interworkings which would allow us to finally import our module.

Let’s use this:

from io.fileops import read_translation

And then of course remove the dot-notation navigation to this function — use it directly:

translation = read_translation(language_code)

All these exercises, however, lead to the same result:

ModuleNotFoundError: No module named 'io.fileops'; 'io' is not a package

The documentation

At this point, it is time to hit the documentation and read carefully about Python’s import system. So I went directly to docs.python.org and looked for “import”.

The reference

This leads me to the following page: 5. Imports. This is the official reference to the import system in Python. Unfortunately, it is not written for beginners. It is a reference, after all, its job is to explain everything in detail, but it also assumes the reader has extensive knowledge of Python. I, on the other hand, am only a beginner. This combination makes the reference a very boring and frustrating read. There is just a single section which I understand and which may be connected with my problem: 5.7 Package Relative Imports But even if this section has something to do with my problem, it is written in such a formal way, that I miss the point in it.

The tutorial

Luckily, docs.python.org also has some helpful tutorials like this one: 6. Modules where I found a very detailed and easy to understand explanation of how to create and use modules, packages and other stuff. This includes the use of imports.

The knowledge

In this tutorial, I read that the import keyword is used for absolute imports, meaning that you have to specify the absolute path to the submodule you want to use. In case you want to use a module from a neighbouring submodule, you can use the other syntax form: from path.to.submodule import thing. Using this form also means you do not need to specify the module name when using the thing because it is added to the name table of your module.

But all this is quite intuitive and it does not tell us what our problem is and how to solve it.

The Package section

Inside this tutorial, however, there is a section describing the structure and interworkings of a package. This includes different ways of performing imports as well. There we can see how exactly we need to specify the module we want to import with the different forms of imports, as well as referencing modules inside of our own package, etc.

A particularly interesting piece of information there is the one about relative and absolute imports.

The solution

The unsuccessful attempts

Knowing about absolute and relative imports, we can try a slightly different from-import:

from .io.fileops import read_translation

But this one fails with an import error:

ImportError: attempted relative import with no known parent package

Reading the tutorial a bit more carefully would show us that a relative import is not allowed in the module with __name__ = '__main__', so this is not the way to go. However it explains why if you try to import a module into the interactive prompt everything is fine, but executing it using python -m would make it fail. Simply the module name changes depending on how you use it.

That means if you ever use a relative import in your module, you cannot later use it as a standalone application.

Clearly using relative imports is not the right way to go here. But what is then making our application not recognize the new io submodule we just created? So we need to go with absolute imports. This means we need to change the import from:

from .io.fileops import read_translation

to:

from awesomepackage.io.fileops import read_translation

The results are unimpressive. The same ModuleNotFoundError pops up on the screen.

Where you run your package from

Up until now, I have been executing the package using: python -m main. This command was executed in the root of our package. But if I instead try to run it from outside the package, I end up with a different result. This time there is an error, but it doesn't look like an import error, it looks like our i18n_en.properties file is missing. I guess the import finally worked!

Of course, this is for the price of ugly execution command: python -m awesomepackage.main

It looks like I’m executing a file with the extension, instead of a python module. However, a quick search points me to moving the entry point of the package to the special file . Now I can execute the package by referencing it instead of its module.

Conclusions

Python has a complex import syntax with many forms and nuances. The different import statements are obviously there to help you in different situations, but for the beginner, they are somewhat perplexing. However, if you read a bit about them, it becomes clear which one is needed where.

Often, when talking about imports, creating a package is at the beginning of this journey. Knowledge of how packages work in Python is undoubtedly fundamental for anyone who wants to create a sophisticated application.

So go on and read a bit and try out what you’ve learned. Then rinse and repeat. That’s how one becomes a master, right? Right?

Cheers,

Originally published at https://geneshki.com on April 30, 2022.