Python's venv

There are several ways to isolate Python at runtime. The simplest and most obvious, although hardest to maintain, is to manually change the PATH and PYTHONPATH environment variables and/or move the Python binary to a different, customized place where we want to store our project's dependencies, in order to affect the way that it discovers available packages. Fortunately, there are several tools available that can help in maintaining the virtual environments and packages that are installed for these environments. These are mainly virtualenv and venv. What they do under the hood is, in fact, the same that we would do manually. The actual strategy depends on the specific tool implementation, but generally they are more convenient to use and can provide additional benefits.

To create new virtual environment, you can simply use the following command:

python3.7 -m venv ENV

Here, ENV should be replaced by the desired name for the new environment. This will create a new ENV directory in the current working directory path. Inside, it will contain a few new directories:

• bin/: This is where the new Python executable and scripts/executables provided by other packages are stored.
• lib/ and include/: These directories contain the supporting library files for new Python inside the virtual environment. The new packages will be installed in ENV/lib/pythonX.Y/site-packages/.

Once the new environment has been created, it needs to be activated in the current shell session using UNIX's source command:

source ENV/bin/activate

This changes the state of the current shell sessions by affecting its environment variables. In order to make the user aware that they have activated the virtual environment, it will change the shell prompt by appending the (ENV) string at its beginning. To illustrate this, here is an example session that creates a new environment and activates it:

$ python -m venv example
$ source example/bin/activate
(example) $ which python
/home/swistakm/example/bin/python 
(example) $ deactivate
$ which python
/usr/local/bin/python

The important thing to note about venv is that it depends completely on its state, as stored on a filesystem. It does not provide any additional abilities to track what packages should be installed in it. These virtual environments are also not portable, and should not be moved to another system/machine. This means that the new virtual environment needs to be created from scratch for each new application deployment. Because of this, there is a good practice that's used by venv users to store all project dependencies in the requirements.txt file (this is the naming convention), as shown in the following code:

# lines followed by hash (#) are treated as a comments 
 
# strict version names are best for reproducibility 
eventlet==0.17.4 
graceful==0.1.1 
 
# for projects that are well tested with different 
# dependency versions the relative version specifiers  
# are acceptable too 
falcon>=0.3.0,<0.5.0 
 
# packages without versions should be avoided unless 
# latest release is always required/desired 
pytz 

With such files, all dependencies can be easily installed using pip, because it accepts the requirements file as its output:

pip install -r requirements.txt

What needs to be remembered is that the requirements file is not always the ideal solution, because it does not define the exact list of dependencies, only those that are to be installed. So, the whole project can work without problems in some development environments but will fail to start in others if the requirements file is outdated and does not reflect the actual state of the environment. There is, of course, the pip freeze command, which prints all packages in the current environment, but it should not be used blindly. It will output everything, even packages that are not used in the project but are installed only for testing.

The differences between venv versus virtualenv are discussed in the next section.