venv and Virtual Environments | Ansys Developer Portal

Introduction

As developers we dislike when things go wrong. The only way we can minimise anything going wrong is if we, personally, take responsibility for every step along the way to our goal. However, this is an extreme position to take and there are no developers (that I'm aware of) that build their own computers and operating systems before they can begin to work on their latest project. A balance has to be struck, and some things are more complicated than others. Where you strike the balance is really about identifying which things are hard and which are easy. Which can you realistically do and which are impossible. In my experience, virtual environments are one of those things that are actually pretty easy, but perceived as very hard by new developers. New devs often turn to GUI-based environment managers like anaconda, or even our own Ansys Python Manager, to avoid learning to work with virtual environments on the command line*. But even if you are going to turn to these GUI-fronted package managers when working with PyAnsys or otherwise, it is useful to know about what's going on under the hood when you use them. These tools will not always be available to you in the future, but so long as you have a computer and an installation of Python you'll have a command line/terminal and the power to make virtual environments. This article is all about that.

Installing Python

When you install Python for the first time as a budding developer everything is free and full of potential. The landscape is vast and offers a wealth of bounties over every hill and in every cleft in the land. What could be hiding round the next corner? It's unknown and exciting.

However, eventually the honeymoon period wears off and you're confronted with the cold reality of programming. Many of those hills hid rivers of blood and those inviting clefts in the land? They were hiding peat bogs, and quicksand. Of course, in the meantime you've also fallen in love with navigating the unforgiving landscape of programming and you live for it. No one ever said development isn't a mixed bag.

So one day, you're traipsing across some new low hills from a package you recently upgraded, exploring some new features deftly swinging over a pit here and fending off a cave troll there (you are naturally using this new landscape to workaround a customer requirement in your latest project). When all of a sudden, a previous customer (who used the same project) calls to tell you they lost their cat in one of the peat bogs, and could you go and fish him out? Your manager agrees and away you go. Unfortunately, you get to the bog and there's no cat! It turns out the peat bog was fixed in your current version. Or at least you think so, because otherwise the cat would be here right? OK so the next step is to install the old version and hunt down this poor cat, except you still need to continue developing your current project, which uses the latest version.

Ploppers.

You're snookered! In fact, after a brief investigation you realise the situation is worse than you thought. You've upgraded several packages since then, including your language version, and any one of these things could affect the cat and its chances of escaping the peat bog. You silently curse this customer and their vagaries with their cat. What to do? Well, there is one solution that you know about but have been putting off learning for some time. One way, you begrudgingly admit, you can solve this issue is with a virtual environment.

A "virtual environment" is an entirely localised install of a language to a particular directory. This install does not interfere with your global installation, and can have its own suite of packages installed independently of your global installation and also of any other virtual environments on the same computer. In other words you can install the older version of your language as well as all the packages you had at the time of distributing the tool to the customer, and their precise versions. This makes it trivial to find the cat and ensure that if you can get it out, the customer will be able too (although the fix may be non-trivial; programming can be like that).

Setting up this environment was a right pain to get exactly right, but it really did solve your problems. At this moment you have a minor epiphany and begin to understand why so many developers use virtual environments all the time. If you made a new environment for every project, you'd never have to recreate one. You could just go back to the old environment and begin fixing things straight away. It would be easy.

In fact, as you turn it over in your mind a number of benefits become clear. Customers always want to know what they have to install when you give them your scripts, and this has historically been a huge pain in the back. You've had to work out all the things you use from the imports in your scripts, and the versions thereof through further analysis of what is installed at that moment. But if every project was in its own virtual environment... You would only ever have to get a list of what was installed. You remember that you've seen the term venv in a few places before this, mainly in PyCharm, the code editor that irritating Python developer (me) won't shut up about. So you venture forth to do some research about this tool; maybe it can really help you in the future.

Creating Virtual Environments Using `venv`

That long preamble brings us to here. How you can make your own virtual environments in Python and leverage them in your own development.

There are many virtual environment-creation packages out there, such as pyenv, virtualenv, and conda, but this article will focus on venv, which is part of the Python standard library and will be available to anyone who has Python installed.

Virtual environments (referred to as venvs hereafter for brevity) are simple to create and the basic way of doing it is through the command line/terminal.

In order to run a Python package on the command line we must call python -m example_module, which runs the library module example_module as a script. Some Python packages are built specifically to be run like this (including venv).

PS C:\Users\jderrick> python -m venv --help
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] [--upgrade] [--without-pip] [--prompt PROMPT] [--upgrade-deps] ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
  ENV_DIR               A directory to create the environment in.

options:
  -h, --help            show this help message and exit
  --system-site-packages
                        Give the virtual environment access to the system site-packages dir.
  --symlinks            Try to use symlinks rather than copies, when symlinks are not the default for the platform.
  --copies              Try to use copies rather than symlinks, even when symlinks are the default for the platform.
  --clear               Delete the contents of the environment directory if it already exists, before environment creation.
  --upgrade             Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place.
  --without-pip         Skips installing or upgrading pip in the virtual environment (pip is bootstrapped by default)
  --prompt PROMPT       Provides an alternative prompt prefix for this environment.
  --upgrade-deps        Upgrade core dependencies: pip setuptools to the latest version in PyPI

Once an environment has been created, you may wish to activate it, e.g. by sourcing an activate script in its bin directory.

Running venv with the --help flag gives us plenty of information about the tool and how to use it. The simple thing to do is just create a new venv, like so.

PS C:\Users\jderrick> python -m venv my_new_venv
PS C:\Users\jderrick>

It takes a couple of seconds to execute, but that's it! We just created our first venv! Now to actually use it we need to go a bit further. We need to "activate" it (and that IS a technical term in this case).

If we have a look at what the command has done, it has created a directory called my_new_venv with the following contents.

PS C:\Users\jderrick> ls .\my_new_venv\

    Directory: C:\Users\jderrick\my_new_venv

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        04/10/2022     15:34                Include
d-----        04/10/2022     15:34                Lib
d-----        04/10/2022     15:34                Scripts
-a----        04/10/2022     15:34            122 pyvenv.cfg

PS C:\Users\jderrick>

And that's it! Now these files & directories all have their own purposes but you don't have to know exactly what they all mean.

Include/ - for files related to C extensions (don't worry about this one)
Lib/ - location of packages you install to that venv
Scripts/ - activation scripts & the actual Python executable
pyenv.cfg - configuration file about the venv.

All you actually need to know is that Scripts/ contains the activation script.

Calling activate within the directory, will activate your venv and allow you access to the Python within.

PS C:\Users\jderrick> .\my_new_venv\Scripts\activate
(my_new_venv) PS C:\Users\jderrick>

You should note that the Python version your venv uses will be the same as the Python you used to create it. If you want to make a venv from a different version, all you need to do is install the version you want somewhere on your computer but DON'T "add to path" as you normally would. You also need to customise the installation so that the version is somewhere you can easily get to. Then you can call the python executable from that installation and make a new venv. I actually have several versions of Python installed on my machine for just this purpose.

Now you have a venv you can continue as normal! pip and such will all work as before. When you're done, you only need to deactivate it. This can be done with the simple deactivate command.

(my_new_venv) PS C:\Users\jderrick> deactivate
PS C:\Users\jderrick>

Super easy!

`venv` and Visual Studio Code

VS Code actually makes things pretty easy for you. Below I open a simple PyCharm project I created with VS Code and run the script using the Python extension. VS Code automatically detects the venv/ directory and uses that version of Python to run the script! Very neat.

You can actually go a bit further with PyCharm though.

`venv` and PyCharm

PyCharm goes a step further and locates existing installations of Python on your computer and offers to create a new venv whenever you create a new project. Pretty neat! In fact with PyCharm you don't have to use the command line at all; it uses venv behind the scenes and happily takes care of everything for you. It's a great product and my favourite IDE.

Conclusion

In conclusion, virtual environments are a valuable tool to use when developing Python. Their value increases with the amount of development work that you do. I recommend starting with the basics and working your way up. Start with venv because it is builtin and will always be there for you, and then work your way up to other tools as your experience builds.

TL;DR

Setting up environments is difficult and sharing them is hard
Consistent environments are essential for correct deployment of develop and production workspaces
Putting the effort into creating a virtual environment will reduce the chance of errors when setting up environments
I recommend learning Python's builtin venv package first because it will always be there for you.

Useful Articles/Further Reading

* Aside on the command line vs GUIs

To be clear, I am not saying those GUI-based venv tools are bad (they are not), but that as developers we should at least know the basics of what's going on under the hood. I suspect the aversion from the command line may stem from its strange pop culture depiction of being the realm of "hackers" and "computer wizz"s. It looks alien and scary, usually jet black by default, and just a few extra characters and you can accidentally cause catastrophic damage to your computer. Which I think is kind of a shame as it's not really very scary and actually shockingly useful. Plus you can still accidentally do massive damage to your computer through the GUI; the command line isn't inherently more dangerous. Computers are capable tools and should be treated kindly and with respect!

Introduction

Installing Python

Creating Virtual Environments Using venv

venv and Visual Studio Code

venv and PyCharm