This post illustrates how you can enable Python virtualenv in GCP JupyterLab so that you can organize your .ipynb files to use different virtual environments to keep track of Python package dependencies.
- You are using GCP JupyterLab.
- You want to adhere to the Python development best practices by not polluting the global environment with your Python packages so that you can generate a cleaner “pip freeze” in the future.
- You want each Notebook file (.ipynb) to have its own environment so that you can run them with different package versions.
- You configured a Python virtual environment, but “pip install” from the Notebook file still installs the packages in the global environment.
- You are fed up.
Configuring Virtual Environments
In the JupyterLab Notebook’s terminal, create an empty directory to organize all virtual environments.
Ensure ipykernel is installed. This is used to create new IPython kernels.
python3 -m pip install ipykernel
For each new virtual environment, run the following commands to perform these steps:
- Get into the base virtual environment directory.
- Define a new virtual environment name. Replace [NEW_ENV_NAME] with a new name.
- Create new Python virtual environment.
- The –system-site-packages option ensures you can still use the “data-sciencey” packages that come pre-installed with the GCP JupyterLab Notebook within your new virtual environment.
- Jump into the newly created virtual environment.
- Create a new IPython kernel.
- Exit from virtual environment.
cd virtualenv VENV=[NEW_ENV_NAME] # Update this! python3 -m venv $VENV --system-site-packages source $VENV/bin/activate python -m ipykernel install --user --name=$VENV deactivate
Configuring Notebook File
Create a new Notebook file (.ipynb).
In Select Kernel dialog, select the kernel that you created. In this example, there are 2 new virtual environments (“smurfs” and “thundercats”).
To perform a simple test, install a new package.
%pip install pandas==1.3.0
IMPORTANT: You need to use IPython’s Magics (literally speaking) to ensure the packages are installed in the virtual environment.
- %pip = This uses the pip package manager within the current kernel. Magic!
- ! pip = This uses the pip package manager from the underlying OS. No magic!
From the menu bar, select Kernel -> Restart Kernel and Clear All Outputs… . Always restart the kernel when new packages are installed with %pip.
Inspect the package version. This should show the version you have just installed.
import pandas print(pandas.__version__)
To verify this actually works, create another Notebook file pointing to another kernel. In this file, install the same package but with different version.