Software: Python (Python2 and Python3)
Python is a popular interpreted programming language. Talapas has a large number of Python instances installed, but choosing the best one for your need can be a bit daunting. Here is some guidance.
Python 2 vs Python 3
Although Python3 was derived from Python2, it’s best to think of them as two distinct and incompatible languages. These days, Python2 is deprecated, and it’s best not to use it for new work unless you really need to (perhaps because a module you need is not available for Python3).
In any case, you will want to choose the correct one for your need, as they are not interchangeable at all.
As a detail, always use the commands ‘python3’ and ‘pip3’ when using Python3, and ‘python2’ and ‘pip2’ when using Python2. Historically, there were commands ‘python’ and ‘pip’ that more or less mean “the version of Python I have installed or that I intend”. But these days, they are ambiguous and deprecated.
There Can Be Only One
In any given shell, you should choose only one of the below alternatives. Trying to load two in the same shell can lead to obscure crashes and other bugs. You can always unload everything with
module purge
but you might be better off just starting a new shell.
Versions within Versions
Unfortunately, each Python installation typically has a significant number of Python packages installed within it (typically from PyPI or an Anaconda repo). Although we would prefer to create a Python install with all versions set and never modify it again, for reproducibility, this would in practice lead to an explosion of versions. (And as you can see, we have many already.)
Instead, we occasionally update a minimal set of installed packages upon user request, if it seems that this can be done without disrupting other users. With Miniconda (see below), we often just create a new environment to avoid such issues.
Environment Modules
If you just want quick access to a somewhat generic Python install, you can get one with one of these commands
module load python3
or
module load python2
You can also look for versions newer than the default with commands like
and so on.
Note that each version has a distinct set of “add-on” modules that have been installed via ‘pip3’ (or ‘pip2’). We can add modules by request, but often it might be easier to find a version that already has them.
Miniconda
This is our “go to” solution for most new Python installs these days.
We have a ‘miniconda’ module, which contains within it a large number of environments, almost all of which have a ‘python’ installed. Some of these are the instances that sit behind our Open OnDemand Jupyter instance, so you might be using one already without realizing it.
The most used one currently can be loaded with
which is equivalent to
The name is a bit of an artifact, but refers to the fact that TensorFlow version 1 is included. It also has a large number of other data packages like ‘pandas’, ‘numpy’, ‘scipy’, etc.
A similar module is
which is equivalent to
This is one recommended alternative, if you’re not sure how to start.
It’s quite similar, except that it has TensorFlow version 2. (Note that version 1 is largely deprecated upstream at this point.)
The above choices are stable, but also a bit dated, having been created in mid-2020 or so.
If you’d like to use a similar but quite up-to-date environment, you can do this
This is the module that currently the new Open OnDemand Jupyter instance uses. (Look for the one with the similar date.) This is still in testing and might change a bit, but it’s pretty stable.
An Environment of One’s Own
When we get requests to install combinations of packages that might disrupt other users, we often just create a custom conda environment. All of these are visible to all users (see ‘conda env list’), but their contents might be obscure. In any case, if you have a particular set of packages and versions that you need, we can often provide a custom environment for you (which will also be available to other Talapas users).
Out With the Old
There are a number of other possibilities available, but you probably don’t want to use them, unless perhaps your instructor has recommended them.
RHEL Distribution
If you have no modules loaded, you’ll see ‘python3’ and ‘python2’ from the underlying RedHat distribution. These typically live in ‘/usr/bin/python3' and so on. If you need an exceedingly generic Python, these might be fine. But, we do not recommend them, nor install additional packages. These get updated as we update RHEL.
Anaconda Modules
There are environment modules ‘anaconda2’ and ‘anaconda3’. These were from an early attempt to get Anaconda working on Talapas. You can still use them, but they are not recommended, and we no longer update them. These are superseded by the ‘miniconda’ module.
EasyBuild
There are some versions of Python available via the ‘easybuild’ module. Unless you are in a class or lab that recommends them, you should not use them.
Similarly, there is a ‘racs-eb’ module. Same drill--avoid unless told otherwise.
Spack
Spack (available via ‘racs-spack’) is great, but we generally do not support Python this way. You can try them, but avoid unless told otherwise. We usually don’t add modules except to support software that requires it.
Singularity
Some Singularity images have Python inside. If this works for you, it’s great. If you don’t understand this option, it’s likely not for you.
Go Your Own Way
You can install your own Python: modules or entire instance. This might be good in that it’s entirely under your control. But, support from RACS will be more limited.
Just Modules
In principle, for any of the above solutions, you could layer on code of your own by modifying ‘PYTHONPATH’, or perhaps by using the “per-user” flags for ‘pip’ or ‘conda install’. Some users make this work, but there are dragons here, especially with the latter options. It’s easy to create a combination that fails obscurely. If you’re a relatively novice user, you might do better to ask us for what you want.
Entire Instance
In this case, you’d install Python from scratch, manage it, install your own packages, etc. You might do this from source, or perhaps you’d install your own Anaconda instance. If you’re up to it, this might be a great idea. But, mixing and matching with the above Talapas-provided solutions likely won’t end well. Don’t cross the streams.