PROBLEM

• You are using GCP JupyterLab.
• You want to adhere to the Python development best practices by not polluting the global environment with your Python packages so that you can generate a cleaner “pip freeze” in the future.
• You want each Notebook file (.ipynb) to have its own environment so that you can run them with different package versions.
• You configured a Python virtual environment, but “pip install” from the Notebook file still installs the packages in the global environment.
• You are fed up.

SOLUTION

Configuring Virtual Environments

In the JupyterLab Notebook’s terminal, create an empty directory to organize all virtual environments.

mkdir virtualenv

Ensure ipykernel is installed. This is used to create new IPython kernels.

python3 -m pip install ipykernel

For each new virtual environment, run the following commands to perform these steps:

• Get into the base virtual environment directory.
• Define a new virtual environment name. Replace [NEW_ENV_NAME] with a new name.
• Create new Python virtual environment.
  • The –system-site-packages option ensures you can still use the “data-sciencey” packages that come pre-installed with the GCP JupyterLab Notebook within your new virtual environment.
• Jump into the newly created virtual environment.
• Create a new IPython kernel.
• Exit from virtual environment.

cd virtualenv
VENV=[NEW_ENV_NAME] # Update this!
python3 -m venv $VENV --system-site-packages
source $VENV/bin/activate
python -m ipykernel install --user --name=$VENV
deactivate

Configuring Notebook File

Create a new Notebook file (.ipynb).

In Select Kernel dialog, select the kernel that you created. In this example, there are 2 new virtual environments (“smurfs” and “thundercats”).

To perform a simple test, install a new package.

%pip install pandas==1.3.0

IMPORTANT: You need to use IPython’s Magics (literally speaking) to ensure the packages are installed in the virtual environment.

• %pip = This uses the pip package manager within the current kernel. Magic!
• ! pip = This uses the pip package manager from the underlying OS. No magic!

From the menu bar, select Kernel -> Restart Kernel and Clear All Outputs… . Always restart the kernel when new packages are installed with %pip.

Inspect the package version. This should show the version you have just installed.

import pandas
print(pandas.__version__)

To verify this actually works, create another Notebook file pointing to another kernel. In this file, install the same package but with different version.

The post Enabling Python VirtualEnv in JupyterLab appeared first on My Shitty Code.