Table of Contents

Why Cache Access Token with MSAL?

Building upon the previous post that performs delegated access authentication with MSAL, suppose your program uses this function to get the access token.

def get_access_token():
    app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
    result = app.acquire_token_interactive(scopes=SCOPES)
    access_token = result['access_token']

    return access_token

In this case, it will always launch the browser to complete the authentication flow and retrieve the access token every time the program runs.

While it works, it introduces a slight latency to your program. Fortunately, MSAL provides the capability to cache the access token.

Solution A: Use custom persistence cache

In this solution, we can introduce our custom persistence cache.

First, we can use these functions to load the access token from the cache file and save it to the cache file.

import os
import msal

...
CACHE_FILE = 'msal_token_cache.json'

def load_cache():
    cache = msal.SerializableTokenCache()

    if os.path.exists(CACHE_FILE):
        with open(CACHE_FILE, 'rb') as f:
            cache.deserialize(f.read())

    return cache

def save_cache(cache):
    if cache.has_state_changed:
        with open(CACHE_FILE, 'wb') as f:
            f.write(cache.serialize().encode())

We can first attempt to get a valid access token from the cache. It will only get the token interactively via the browser if the access token doesn’t exist or has expired.

def get_access_token():
    cache = load_cache()
    app = msal.PublicClientApplication(
        CLIENT_ID,
        authority=AUTHORITY,
        token_cache=cache,
    )
    result = None
    accounts = app.get_accounts()

    if accounts:
        result = app.acquire_token_silent(SCOPES, account=accounts[0])

    if not result:
        result = app.acquire_token_interactive(scopes=SCOPES)

    save_cache(cache)
    access_token = result['access_token']

    return access_token

While this solution works, the implementation code is very lengthy. Furthermore, the access token is stored in the cache file as clear text.

Solution B: Use the OS platform’s encrypted vault (PREFERRED)

A Python package called msal-extensions allows the access token to be stored securely in the OS platform’s encrypted vault. For example, If the program runs on MacOS, KeyChain will be used. Windows uses DPAPI, and Linux uses LibSecret.

To begin, install that Python package.

# requirements.txt

msal
msal-extensions

The configuration is much simpler than Solution A.

import msal
from msal_extensions import build_encrypted_persistence, PersistedTokenCache

...
CACHE_FILE = 'msal_token_cache.json'

def get_access_token():
    app = msal.PublicClientApplication(
        CLIENT_ID,
        authority=AUTHORITY,
        token_cache=PersistedTokenCache(build_encrypted_persistence(CACHE_FILE)),
    )

    result = None
    accounts = app.get_accounts()

    if accounts:
        result = app.acquire_token_silent(SCOPES, account=accounts[0])

    if not result:
        result = app.acquire_token_interactive(scopes=SCOPES)

    access_token = result['access_token']

    return access_token

When running the program on MacOS, the access token is stored in the KeyChain.

Note: An empty cache file named msal_token_cache.json has also been created. Unfortunately, there is no option in msal-extensions==1.2.0 to disable the creation of this file at the moment.

The post MSAL: Caching Access Token appeared first on My Shitty Code.

Table of Contents

Why Launch a Specific Browser with MSAL?

Building upon the previous post that performs delegated access authentication with MSAL, your institution may sometimes allow you to access Microsoft 365 products only on the approved browsers.

By default, MSAL launches the default browser on your machine to obtain the access token interactively.

If your institution only allows Chrome and your default browser is Firefox, you will get the following message:

Overriding with BROWSER environment variable

MSAL relies on Python’s built-in module called webbrowser to launch the browser. This allows us to change the browser by setting the BROWSER environment variable to the predefined values (‘firefox’, ‘chrome’, etc.)

For example, to launch Firefox instead of the default browser, we can do this:

import os
import msal

# ... 
os.environ['BROWSER'] = 'firefox'
app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
result = app.acquire_token_interactive(scopes=SCOPES)

What if the BROWSER environment variable’s predefined value doesn’t work?

I discovered that Python’s webbrowser failed to launch the correct browser, especially when dealing with Chromnium-based browsers.

For example, suppose my default browser is Brave, and I set the BROWSER environment variable to Chrome. In that case, it will still launch the Brave browser.

# default browser = Brave

os.environ['BROWSER'] = 'chrome'  # doesn't work
app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
result = app.acquire_token_interactive(scopes=SCOPES)

Fortunately, it is possible to specify a full path ending with ‘%s’ instead:

# default browser = Brave

os.environ['BROWSER'] = 'open -a /Applications/Google\\ Chrome.app %s'  # works
app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
result = app.acquire_token_interactive(scopes=SCOPES)

If you are stuck on the “Approve sign in request” or “Set up your device to get access” screen indefinitely, consider clearing the browser cache and then try again.

The post MSAL: Launching Specific Browser with Delegated Access Authentication appeared first on My Shitty Code.

Table of Contents

The Microsoft Authentication Library (MSAL) supports various programming languages and frameworks to simplify the authentication flow against the Microsoft identity platform, which is a prerequisite to invoke APIs such as Microsoft Graph.

In this example, we will write a Python script that performs delegated authentication flow using the user’s credentials. Once authenticated, the script will pull the user’s OneNote data via Microsoft Graph.

Note: While we can also authenticate via app-only access, this is generally scrutinized in larger institutions due to the blast radius. This is because the app can modify other users’ content stored in Microsoft 365 products (emails, calendars, etc).

App Registration and Configuration

Even if you plan to run your program on your local machine, you must register it in the Azure Portal to generate a unique client ID to authenticate.

• Go to https://portal.azure.com/ and sign in.
• Go to App Registrations and register your app.
• Go to the registered app -> Manage (step 1) -> Authentication (step 2). Ensure the following configurations exist:
  • A Mobile and desktop applications platform with the redirect URI set as http://localhost (step 3).
  • The Access tokens (used for implicit flows) option is checked (step 4).

• Go to the registered app -> Manage (step 1) -> API permissions (step 2).
• Add the permissions you need (step 3). OneNote’s permissions are added in this example because the script will query OneNote via Microsoft Graph.
• Before we can call the Microsoft Graph API, these permissions must have green check marks (step 5), accomplished by clicking the Grant admin consent button (step 4). If this button is grayed out, request your institution’s Azure administrators to perform this step on your behalf.

Custom Python Program

Now that the app is registered and configured in the Azure Portal, we can work on the custom script.

Install the msal library.

# requirements.txt

requests
msal

Define the needed variables. The client ID and tenant ID can be retrieved from your registered app in the Azure Portal.

import msal
import requests

CLIENT_ID = '[YOUR_CLIENT_ID]'
TENANT_ID = '[TENANT_ID]'
AUTHORITY = f'https://login.microsoftonline.com/{TENANT_ID}'
SCOPES = ['Notes.Read', 'User.Read']

A function to perform the authentication and to retrieve the access token. MSAL will seamlessly create a web server running on localhost to accept the incoming token data (hence, the need to configure the redirect URI in the Azure Portal).

def get_access_token():
    app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
    result = app.acquire_token_interactive(scopes=SCOPES)
    access_token = result['access_token']

    return access_token

A simple function to query OneNote pages using Microsoft Graph.

def get_onenote_pages(access_token):
    url = 'https://graph.microsoft.com/v1.0/me/onenote/pages'

    headers = {
        'Authorization': f'Bearer {access_token}',
        'Accept': 'application/json'
    }

    response = requests.get(url, headers=headers)
    response.raise_for_status()

    return response.json()

Finally, run the app to test if the app registration and configuration are set up correctly in the Azure Portal.

def run():
    access_token = get_access_token()
    print(access_token)

    onenote_pages = get_onenote_pages(access_token)
    print(onenote_pages)


if __name__ == '__main__':
    run() 

The post MSAL: Delegated Access Authentication appeared first on My Shitty Code.

Table of Contents

There are various ways for a Cloud Run instance to connect to a Cloud SQL instance. Regardless of the solutions, if both resources reside in the same GCP organization, connecting using a Cloud SQL’s private IP with SSL/TLS encryption is highly recommended to prevent network traffic from going out to the internet.

It is possible to have a Cloud SQL instance with both private and public IPs enabled, and there are legitimate reasons for doing so. For example, it allows the developers to connect to the Cloud SQL instance outside the GCP environment. However, utmost care is needed to ensure the private IP is always used when connecting from the Cloud Run instance.

This article evaluates the pros and cons of each solution and provides some recommendations.

Cloud SQL Configuration

For consistency’s sake, SSL/TLS encryption is enabled on the Cloud SQL instance using client certificates.

A client certificate is also created.

Note: Some solutions, such as Cloud SQL Proxy and Cloud SQL Language Connectors, do not need the client certificate. So, these steps can be safely skipped. From the documentation:

With Cloud SQL Auth Proxy and Cloud SQL Connectors, client and server identities are also automatically verified regardless of the SSL mode setting.

Options

Option A: Cloud SQL Language Connectors

This solution uses Cloud SQL Language Connectors, a programming language library maintained by Google.

The following configurations are NOT required:

• SSL certificates in the Cloud SQL instance
• Cloud SQL volume mounting in the Cloud Run instance.

Python App

import os

from google.cloud.sql.connector import Connector, IPTypes

# ... simplified for brevity

connector = Connector(IPTypes.PRIVATE)  # use Private IP

connector.connect(
  os.getenv('DB_INSTANCE_CONNECTION_NAME'),  # project:region:instance
  'pymysql',
  user=os.getenv('DB_USER'),
  password=os.getenv('DB_PASS'),
  db=os.getenv('DB_NAME'),
)

Pros

• Very concise configuration in both IaC and application code since there’s no need to manage client certificates.

Cons

• Only supports Python, Java, Go, and Node.js.

Option B: Managed Cloud SQL Proxy in Cloud Run

This solution uses a managed Cloud SQL Proxy that is configurable directly in the Cloud Run instance. Once configured, a Cloud SQL volume mount is created in the Cloud Run instance.

Python App

import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  database=os.getenv('DB_NAME'),
  user=os.getenv('DB_USER'),
  password=os.getenv('DB_PASS'),
  unix_socket=f'/cloudsql/{os.getenv("DB_INSTANCE_CONNECTION_NAME")}',  # /cloudsql/project:region:instance
)

Pros

• Very concise configuration in both IaC and application code since there’s no need to manage SSL certificates.

Cons

• It doesn’t work if the Cloud SQL instance has both public IP and private IP enabled. There’s no way to specify which IP to use, and Cloud SQL Proxy automatically uses the public IP if it exists.
  • Note: If you know how to pull this off, please drop a comment below.
• Additional configurations must be considered when running in production, which introduces many points of failure.
• It may exceed the default Cloud SQL Admin API quota limit.

Option C: Manually Configure Cloud SQL Proxy

This solution manually configures Cloud SQL Proxy via Dockerfile instead of relying on the managed Cloud SQL volume mount from Cloud Run.

Dockerfile

FROM python:3.12-alpine

RUN apk update && apk add curl

WORKDIR /app

COPY . /app

RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8080

RUN curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.12.0/cloud-sql-proxy.linux.amd64
RUN chmod +x cloud-sql-proxy

CMD ["sh", "-c", "./cloud-sql-proxy --private-ip --port 3306 ${DB_INSTANCE_CONNECTION_NAME} & sleep 5 && python app.py"]

Python App

import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  host=os.getenv('DB_HOST'),  # 127.0.0.1
  database=os.getenv('DB_NAME'),
  user=os.getenv('DB_USER'),
  password=os.getenv('DB_PASS'),
)

Pros

• Very concise configuration in both IaC and application code since there’s no need to manage client certificates.
• Unlike the managed Cloud SQL Proxy volume mount solution, this solution will work even if the Cloud SQL instance has both private and public IPs.

Cons

• Need to manage the Cloud SQL Proxy installation and configuration in the Dockerfile.
• All the cons described in the managed Cloud SQL Proxy solution above still apply here.

Option D: Use Cloud SQL’s Private IP and Client Certificate

Instead of relying on external tools/libraries, the application code connects directly to the Cloud SQL instance using the private IP where the client certificate is also provided.

Python App

import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  host=os.getenv('DB_HOST'), # Cloud SQL's Private IP
  database=os.getenv('DB_NAME'),
  user=os.getenv('DB_USER'),
  password=os.getenv('DB_PASS'),
  ssl={
    'ca': os.getenv('DB_SSL_SERVER_CA'),     # /app/certs/ca/server-ca.pem
    'cert': os.getenv('DB_SSL_CLIENT_CERT'), # /app/certs/cert/client-cert.pem
    'key': os.getenv('DB_SSL_CLIENT_KEY'),   # /app/certs/key/client-key.pem
    'check_hostname': False,
  }
)

Note: The hostname checking must be disabled because the database host (Cloud SQL’s Private IP) will never
match the common name specified in the certificate. If the hostname checking is not disabled, it will produce the following error:

Error connecting to database: (2003, “Can’t connect to MySQL server on ’10.x.x.x’ ([SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: IP address mismatch, certificate is not valid for ’10.x.x.x’. (_ssl.c:1000))”)

Pros

• No additional wrappers like Cloud SQL Proxy or Cloud SQL Language Connectors are required.

Cons

• Additional configurations and complexity on the IaC and application code are needed to manage and ingest the client certificate securely.

Recommendations

Best Solution – Option A

Use Cloud SQL Language Connectors if your app uses one of the supported programming languages (Python, Java, Go, or Node.js).

It eliminates many unnecessary configurations on the IaC code and application code. In other words, there’s no need to do the following:

• Create a client certificate for Cloud SQL.
• Store server certificate, client certificate, and client key in Secret Manager securely.
• Mount files containing the server certificate, client certificate, and client key as volumes in the Cloud Run instance.
• Pass certificate information to the application code via environment variables.

None of the above is needed.

Less codebase = Less chance of errors.

Distance Next Best Solution – Option D

Connect directly using the Cloud SQL instance’s Private IP and client certificate.

It eliminates the need to rely on additional cloud configuration or external library (i.e., fewer points of failure).

However, it introduces additional configurations in IaC and application code to maintain and ingest the certificate described above.

Least Favorite Solution – Option B & C

While both Cloud SQL Proxy solutions (volume mount in the Cloud Run instance or installing it directly via Dockerfile) don’t require a client certificate, they come with additional considerations and complexities when running in production.

Furthermore, the volume mount solution won’t work if the Cloud SQL instance has both private and public IPs.

The post Cloud Run: Connecting to Cloud SQL with Private IP and SSL appeared first on My Shitty Code.

PROBLEM

• You are using GCP JupyterLab.
• You want to adhere to the Python development best practices by not polluting the global environment with your Python packages so that you can generate a cleaner “pip freeze” in the future.
• You want each Notebook file (.ipynb) to have its own environment so that you can run them with different package versions.
• You configured a Python virtual environment, but “pip install” from the Notebook file still installs the packages in the global environment.
• You are fed up.

SOLUTION

Configuring Virtual Environments

In the JupyterLab Notebook’s terminal, create an empty directory to organize all virtual environments.

mkdir virtualenv

Ensure ipykernel is installed. This is used to create new IPython kernels.

python3 -m pip install ipykernel

For each new virtual environment, run the following commands to perform these steps:

• Get into the base virtual environment directory.
• Define a new virtual environment name. Replace [NEW_ENV_NAME] with a new name.
• Create new Python virtual environment.
  • The –system-site-packages option ensures you can still use the “data-sciencey” packages that come pre-installed with the GCP JupyterLab Notebook within your new virtual environment.
• Jump into the newly created virtual environment.
• Create a new IPython kernel.
• Exit from virtual environment.

cd virtualenv
VENV=[NEW_ENV_NAME] # Update this!
python3 -m venv $VENV --system-site-packages
source $VENV/bin/activate
python -m ipykernel install --user --name=$VENV
deactivate

Configuring Notebook File

Create a new Notebook file (.ipynb).

In Select Kernel dialog, select the kernel that you created. In this example, there are 2 new virtual environments (“smurfs” and “thundercats”).

To perform a simple test, install a new package.

%pip install pandas==1.3.0

IMPORTANT: You need to use IPython’s Magics (literally speaking) to ensure the packages are installed in the virtual environment.

• %pip = This uses the pip package manager within the current kernel. Magic!
• ! pip = This uses the pip package manager from the underlying OS. No magic!

From the menu bar, select Kernel -> Restart Kernel and Clear All Outputs… . Always restart the kernel when new packages are installed with %pip.

Inspect the package version. This should show the version you have just installed.

import pandas
print(pandas.__version__)

To verify this actually works, create another Notebook file pointing to another kernel. In this file, install the same package but with different version.

The post Enabling Python VirtualEnv in JupyterLab appeared first on My Shitty Code.