Using Conda/Anaconda in HPC environment
To use conda, please load modules with command:
module load container_env python3
mkdir -p ~/envs
crun -c -s -p ~/envs/PROJECT_NAME
Please define a sensible PROJECT_NAME for the project you are working on, and we recommand you create a new environment for a new project that uses unrelated libraries from the other project.
To use the enviroment, please prepend your command with "crun -p ~/envs/PROJECT_NAME", for example:
crun -p ~/envs/PROJECT_NAME XXX
crun -p ~/envs/PROJECT_NAME python xxxx.py # if python is installed
crun -p ~/envs/PROJECT_NAME Rscript xxxx.R # if R is installed
crun -p ~/envs/PROJECT_NAME conda search XXX
crun -p ~/envs/PROJECT_NAME conda search -c conda-forge XXX # using additional channel
crun -p ~/envs/PROJECT_NAME conda install XXX
crun -p ~/envs/PROJECT_NAME conda install -c conda-forge XXX # using additional channel
You can use the environment in a job script using similar way, for example:
#!/bin/bash
#SBATCH -c 4
## ... add other SBATCH options here as needed
module load container_env python3
crun -p ~/envs/PROJECT_NAME python3 script.py
Please do not create new environment, install package in job script, it's best you do it in interactive mode and only using the installed package in job script.
If you wish to use Jupyter from https://ondemand.wahab.hpc.odu.edu/, please install "jupyterlab" package:
crun -p ~/envs/PROJECT_NAME conda install -c conda-forge jupyterlab
crun -p ~/envs/PROJECT_NAME conda install -c conda-forge widgetsnbextension ipywidgets # optional if you wish to install additional jupyter extensions, ignore if unsure
You may see instruction provided by online tutoring/documentation that is not work directly in ODU HPC cluster. A simple translatation is required :
¶ Common Conda:
conda create --name snowflakes biopython
module load container_env python3 # you only need do this once per shell session
crun -c -s -p ~/envs/snowflakes
crun -p ~/envs/snowflakes conda install biopython
# or
crun conda create -p ~/envs/snowflakes biopython
¶ Common Conda:
conda env create -n snowflakes --file ENV.yml
Conda in ODU HPC:
crun conda env create -p ~/envs/snowflakes --file ENV.yml
¶ Common Conda:
conda search beautifulsoup4 conda install beautifulsoup4 conda list
Conda in ODU HPC:
crun -p ~/envs/snowflakes conda search beautifulsoup4
crun -p ~/envs/snowflakes conda install beautifulsoup4
crun -p ~/envs/snowflakes conda list
¶ Common Conda:
conda activate snowflakes python xxxx.py
Conda in ODU HPC:
crun -p ~/envs/snowflakes python xxxx.py