<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Programming Language &#8211; My Shitty Code</title>
	<atom:link href="https://myshittycode.com/programming-language/feed/" rel="self" type="application/rss+xml" />
	<link>https://myshittycode.com</link>
	<description>Embracing the Messiness in Search of Epic Solutions</description>
	<lastBuildDate>Mon, 23 Sep 2024 20:17:20 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=6.9.1</generator>

<image>
	<url>https://myshittycode.com/wp-content/uploads/2022/04/cropped-icon-32x32.png</url>
	<title>Programming Language &#8211; My Shitty Code</title>
	<link>https://myshittycode.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">205304208</site>	<item>
		<title>MSAL: Delegated Access Authentication</title>
		<link>https://myshittycode.com/2024/09/23/msal-delegated-access-authentication/</link>
					<comments>https://myshittycode.com/2024/09/23/msal-delegated-access-authentication/#respond</comments>
		
		<dc:creator><![CDATA[Shitty Author]]></dc:creator>
		<pubDate>Mon, 23 Sep 2024 20:17:14 +0000</pubDate>
				<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Programming Language]]></category>
		<category><![CDATA[Azure]]></category>
		<category><![CDATA[MS 365]]></category>
		<category><![CDATA[MSAL]]></category>
		<category><![CDATA[Python]]></category>
		<guid isPermaLink="false">https://myshittycode.com/?p=2678</guid>

					<description><![CDATA[<p>The Microsoft Authentication Library (MSAL) supports various programming languages and frameworks to simplify the authentication flow against the Microsoft identity platform, which is a prerequisite to invoke APIs such as Microsoft Graph. In this example, we will write a Python script that performs delegated authentication flow using the user&#8217;s credentials. Once authenticated, the script will [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2024/09/23/msal-delegated-access-authentication/">MSAL: Delegated Access Authentication</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<div class="wp-block-rank-math-toc-block" id="rank-math-toc"><h2>Table of Contents</h2><nav><ul><li><a href="#app-registration-and-configuration">App Registration and Configuration</a></li><li><a href="#custom-python-script">Custom Python Program</a></li></ul></nav></div>



<p>The <a href="https://learn.microsoft.com/en-us/entra/msal/overview" target="_blank" rel="noopener">Microsoft Authentication Library (MSAL)</a> supports various programming languages and frameworks to simplify the authentication flow against the Microsoft identity platform, which is a prerequisite to invoke APIs such as Microsoft Graph.</p>



<p>In this example, we will write a Python script that performs delegated authentication flow using the user&#8217;s credentials. Once authenticated, the script will pull the user&#8217;s OneNote data via Microsoft Graph.</p>



<p><strong>Note:</strong> While we can also authenticate via app-only access, this is generally scrutinized in larger institutions due to the blast radius. This is because the app can modify other users&#8217; content stored in Microsoft 365 products (emails, calendars, etc).</p>



<h2 class="wp-block-heading" id="app-registration-and-configuration">App Registration and Configuration</h2>



<p>Even if you plan to run your program on your local machine, you must register it in the Azure Portal to generate a unique client ID to authenticate.</p>



<ul class="wp-block-list">
<li>Go to <a href="https://portal.azure.com/" target="_blank" rel="noopener">https://portal.azure.com/</a> and sign in.</li>



<li>Go to <strong>App Registrations</strong> and register your app.</li>



<li>Go to the registered app -> <strong>Manage</strong> (<em>step 1</em>) -> <strong>Authentication</strong> (<em>step </em>2). Ensure the following configurations exist:
<ul class="wp-block-list">
<li>A <strong>Mobile and desktop applications</strong> platform with the redirect URI set as <strong>http://localhost</strong> (<em>step </em>3).</li>



<li>The <strong>Access tokens (used for implicit flows)</strong> option is checked (<em>step </em>4).</li>
</ul>
</li>
</ul>



<figure class="wp-block-image size-full"><img fetchpriority="high" decoding="async" width="2114" height="2908" src="https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication.png?x45560" alt="" class="wp-image-2680" srcset="https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication.png 2114w, https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication-218x300.png 218w, https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication-744x1024.png 744w, https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication-768x1056.png 768w, https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication-1117x1536.png 1117w, https://myshittycode.com/wp-content/uploads/2024/09/msal-authentication-1489x2048.png 1489w" sizes="(max-width: 2114px) 100vw, 2114px" /></figure>



<ul class="wp-block-list">
<li>Go to the registered app -> <strong>Manage</strong> (<em>step 1</em>) -> <strong>API permissions</strong> (<em>step </em>2).</li>



<li>Add the permissions you need (<em>step </em>3). OneNote&#8217;s permissions are added in this example because the script will query OneNote via Microsoft Graph. </li>



<li>Before we can call the Microsoft Graph API, these permissions must have green check marks (<em>step </em>5), accomplished by clicking the <strong>Grant admin consent</strong> button (<em>step </em>4). If this button is grayed out, request your institution&#8217;s Azure administrators to perform this step on your behalf.</li>
</ul>



<figure class="wp-block-image size-full"><img decoding="async" width="2110" height="1484" src="https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions.png?x45560" alt="" class="wp-image-2681" srcset="https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions.png 2110w, https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions-300x211.png 300w, https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions-1024x720.png 1024w, https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions-768x540.png 768w, https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions-1536x1080.png 1536w, https://myshittycode.com/wp-content/uploads/2024/09/msal-permissions-2048x1440.png 2048w" sizes="(max-width: 2110px) 100vw, 2110px" /></figure>



<h2 class="wp-block-heading" id="custom-python-script">Custom Python Program</h2>



<p>Now that the app is registered and configured in the Azure Portal, we can work on the custom script.</p>



<p>Install the msal library.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
# requirements.txt

requests
msal
</pre></div>


<p>Define the needed variables. The client ID and tenant ID can be retrieved from your registered app in the Azure Portal.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
import msal
import requests

CLIENT_ID = &#039;&#x5B;YOUR_CLIENT_ID]&#039;
TENANT_ID = &#039;&#x5B;TENANT_ID]&#039;
AUTHORITY = f&#039;https://login.microsoftonline.com/{TENANT_ID}&#039;
SCOPES = &#x5B;&#039;Notes.Read&#039;, &#039;User.Read&#039;]
</pre></div>


<p>A function to perform the authentication and to retrieve the access token. MSAL will seamlessly create a web server running on localhost to accept the incoming token data (hence, the need to configure the redirect URI in the Azure Portal).</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
def get_access_token():
    app = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY)
    result = app.acquire_token_interactive(scopes=SCOPES)
    access_token = result&#x5B;&#039;access_token&#039;]

    return access_token
</pre></div>


<p>A simple function to query OneNote pages using Microsoft Graph.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
def get_onenote_pages(access_token):
    url = &#039;https://graph.microsoft.com/v1.0/me/onenote/pages&#039;

    headers = {
        &#039;Authorization&#039;: f&#039;Bearer {access_token}&#039;,
        &#039;Accept&#039;: &#039;application/json&#039;
    }

    response = requests.get(url, headers=headers)
    response.raise_for_status()

    return response.json()
</pre></div>


<p>Finally, run the app to test if the app registration and configuration are set up correctly in the Azure Portal.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
def run():
    access_token = get_access_token()
    print(access_token)

    onenote_pages = get_onenote_pages(access_token)
    print(onenote_pages)


if __name__ == &#039;__main__&#039;:
    run() 
</pre></div><p>The post <a rel="nofollow" href="https://myshittycode.com/2024/09/23/msal-delegated-access-authentication/">MSAL: Delegated Access Authentication</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://myshittycode.com/2024/09/23/msal-delegated-access-authentication/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">2678</post-id>	</item>
		<item>
		<title>Cloud Run: Connecting to Cloud SQL with Private IP and SSL</title>
		<link>https://myshittycode.com/2024/08/08/cloud-run-connecting-to-cloud-sql-with-private-ip-and-ssl/</link>
					<comments>https://myshittycode.com/2024/08/08/cloud-run-connecting-to-cloud-sql-with-private-ip-and-ssl/#respond</comments>
		
		<dc:creator><![CDATA[Shitty Author]]></dc:creator>
		<pubDate>Thu, 08 Aug 2024 22:05:43 +0000</pubDate>
				<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Programming Language]]></category>
		<category><![CDATA[Cloud Run]]></category>
		<category><![CDATA[Cloud SQL]]></category>
		<category><![CDATA[Docker]]></category>
		<category><![CDATA[Google Cloud Platform]]></category>
		<category><![CDATA[Python]]></category>
		<guid isPermaLink="false">https://myshittycode.com/?p=2642</guid>

					<description><![CDATA[<p>There are various ways for a Cloud Run instance to connect to a Cloud SQL instance. Regardless of the solutions, if both resources reside in the same GCP organization, connecting using a Cloud SQL&#8217;s private IP with SSL/TLS encryption is highly recommended to prevent network traffic from going out to the internet. It is possible [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2024/08/08/cloud-run-connecting-to-cloud-sql-with-private-ip-and-ssl/">Cloud Run: Connecting to Cloud SQL with Private IP and SSL</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<div class="wp-block-rank-math-toc-block" id="rank-math-toc"><h2>Table of Contents</h2><nav><ul><li><a href="#cloud-sql-configuration">Cloud SQL Configuration</a></li><li><a href="#options">Options</a><ul><li><a href="#option-a-cloud-sql-language-connectors">Option A: Cloud SQL Language Connectors</a></li><li><a href="#option-b-managed-cloud-sql-proxy-in-cloud-run">Option B: Managed Cloud SQL Proxy in Cloud Run</a></li><li><a href="#option-c-manually-configure-cloud-sql-proxy">Option C: Manually Configure Cloud SQL Proxy</a></li><li><a href="#option-d-use-cloud-sq-ls-private-ip-and-client-certificate">Option D: Use Cloud SQL&#8217;s Private IP and Client Certificate</a></li></ul></li><li><a href="#recommendations">Recommendations</a><ul><li><a href="#best-solution-option-a">Best Solution &#8211; Option A</a></li><li><a href="#distance-next-best-solution-option-d">Distance Next Best Solution &#8211; Option D</a></li><li><a href="#least-favorite-solution-option-b-c">Least Favorite Solution &#8211; Option B &amp; C</a></li></ul></li></ul></nav></div>



<p>There are various ways for a Cloud Run instance to connect to a Cloud SQL instance. Regardless of the solutions, if both resources reside in the same GCP organization, connecting using a Cloud SQL&#8217;s private IP with SSL/TLS encryption is highly recommended to prevent network traffic from going out to the internet.</p>



<p>It is possible to have a Cloud SQL instance with both private and public IPs enabled, and there are legitimate reasons for doing so. For example, it allows the developers to connect to the Cloud SQL instance outside the GCP environment. However, utmost care is needed to ensure the private IP is always used when connecting from the Cloud Run instance.</p>



<p>This article evaluates the pros and cons of each solution and provides some recommendations.</p>



<h2 class="wp-block-heading" id="cloud-sql-configuration">Cloud SQL Configuration</h2>



<p>For consistency&#8217;s sake, SSL/TLS encryption is enabled on the Cloud SQL instance using client certificates.</p>



<figure class="wp-block-image size-large is-resized"><img decoding="async" width="1024" height="742" src="https://myshittycode.com/wp-content/uploads/2024/08/ssl-mode-1-1024x742.png?x45560" alt="Cloud SQL" class="wp-image-2651" style="width:800px" srcset="https://myshittycode.com/wp-content/uploads/2024/08/ssl-mode-1-1024x742.png 1024w, https://myshittycode.com/wp-content/uploads/2024/08/ssl-mode-1-300x217.png 300w, https://myshittycode.com/wp-content/uploads/2024/08/ssl-mode-1-768x557.png 768w, https://myshittycode.com/wp-content/uploads/2024/08/ssl-mode-1.png 1104w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>A client certificate is also created. </p>



<figure class="wp-block-image size-large is-resized"><img loading="lazy" decoding="async" width="1024" height="417" src="https://myshittycode.com/wp-content/uploads/2024/08/client-certificate-1-1024x417.png?x45560" alt="" class="wp-image-2650" style="width:800px" srcset="https://myshittycode.com/wp-content/uploads/2024/08/client-certificate-1-1024x417.png 1024w, https://myshittycode.com/wp-content/uploads/2024/08/client-certificate-1-300x122.png 300w, https://myshittycode.com/wp-content/uploads/2024/08/client-certificate-1-768x313.png 768w, https://myshittycode.com/wp-content/uploads/2024/08/client-certificate-1.png 1096w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /></figure>



<p><strong>Note:</strong> Some solutions, such as Cloud SQL Proxy and Cloud SQL Language Connectors, do not need the client certificate. So, these steps can be safely skipped. From the <a href="https://cloud.google.com/sql/docs/mysql/configure-ssl-instance#enforcing-ssl" target="_blank" rel="noopener">documentation</a>:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>With Cloud SQL Auth Proxy and Cloud SQL Connectors, client and server identities are also automatically verified regardless of the SSL mode setting.</p>
</blockquote>



<h2 class="wp-block-heading" id="options">Options</h2>



<h3 class="wp-block-heading" id="option-a-cloud-sql-language-connectors">Option A: Cloud SQL Language Connectors</h3>



<p>This solution uses <a href="https://cloud.google.com/sql/docs/mysql/connect-connectors" target="_blank" rel="noopener">Cloud SQL Language Connectors</a>, a programming language library maintained by Google.</p>



<p>The following configurations are <strong>NOT</strong> required:</p>



<ul class="wp-block-list">
<li>SSL certificates in the Cloud SQL instance</li>



<li>Cloud SQL volume mounting in the Cloud Run instance.</li>
</ul>



<p><strong>Python App</strong></p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
import os

from google.cloud.sql.connector import Connector, IPTypes

# ... simplified for brevity

connector = Connector(IPTypes.PRIVATE)  # use Private IP

connector.connect(
  os.getenv(&#039;DB_INSTANCE_CONNECTION_NAME&#039;),  # project:region:instance
  &#039;pymysql&#039;,
  user=os.getenv(&#039;DB_USER&#039;),
  password=os.getenv(&#039;DB_PASS&#039;),
  db=os.getenv(&#039;DB_NAME&#039;),
)
</pre></div>


<p><strong>Pros</strong></p>



<ul class="wp-block-list">
<li>Very concise configuration in both IaC and application code since there&#8217;s no need to manage client certificates.</li>
</ul>



<p><strong>Cons</strong></p>



<ul class="wp-block-list">
<li>Only supports Python, Java, Go, and Node.js.</li>
</ul>



<h3 class="wp-block-heading" id="option-b-managed-cloud-sql-proxy-in-cloud-run">Option B: Managed Cloud SQL Proxy in Cloud Run</h3>



<p>This solution uses a <a href="https://cloud.google.com/sql/docs/mysql/connect-run#configure" target="_blank" rel="noopener">managed Cloud SQL Proxy</a> that is configurable directly in the Cloud Run instance. Once configured, a Cloud SQL volume mount is created in the Cloud Run instance. </p>



<p><strong>Python App</strong></p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  database=os.getenv(&#039;DB_NAME&#039;),
  user=os.getenv(&#039;DB_USER&#039;),
  password=os.getenv(&#039;DB_PASS&#039;),
  unix_socket=f&#039;/cloudsql/{os.getenv(&quot;DB_INSTANCE_CONNECTION_NAME&quot;)}&#039;,  # /cloudsql/project:region:instance
)
</pre></div>


<p><strong>Pros</strong></p>



<ul class="wp-block-list">
<li>Very concise configuration in both IaC and application code since there&#8217;s no need to manage SSL certificates.</li>
</ul>



<p><strong>Cons</strong></p>



<ul class="wp-block-list">
<li><strong>It doesn&#8217;t work if the Cloud SQL instance has both public IP and private IP enabled</strong>. There&#8217;s no way to specify which IP to use, and Cloud SQL Proxy automatically uses the public IP if it exists.
<ul class="wp-block-list">
<li><strong>Note:</strong> If you know how to pull this off, please drop a comment below.</li>
</ul>
</li>



<li><a href="https://cloud.google.com/sql/docs/mysql/sql-proxy#production-environment" target="_blank" rel="noopener">Additional configurations</a> must be considered when running in production, which introduces many points of failure.</li>



<li>It may exceed the default <a href="https://cloud.google.com/sql/docs/mysql/connect-run#api-quota-limits" target="_blank" rel="noopener">Cloud SQL Admin API quota limit</a>.</li>
</ul>



<h3 class="wp-block-heading" id="option-c-manually-configure-cloud-sql-proxy">Option C: Manually Configure Cloud SQL Proxy</h3>



<p>This solution <a href="https://cloud.google.com/sql/docs/mysql/connect-auth-proxy" target="_blank" rel="noopener">manually configures Cloud SQL Proxy</a> via Dockerfile instead of relying on the managed Cloud SQL volume mount from Cloud Run.</p>



<p><strong>Dockerfile</strong></p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [13,14,16]; title: ; notranslate">
FROM python:3.12-alpine

RUN apk update &amp;amp;&amp;amp; apk add curl

WORKDIR /app

COPY . /app

RUN pip install --no-cache-dir -r requirements.txt

EXPOSE 8080

RUN curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.12.0/cloud-sql-proxy.linux.amd64
RUN chmod +x cloud-sql-proxy

CMD &#x5B;&quot;sh&quot;, &quot;-c&quot;, &quot;./cloud-sql-proxy --private-ip --port 3306 ${DB_INSTANCE_CONNECTION_NAME} &amp;amp; sleep 5 &amp;amp;&amp;amp; python app.py&quot;]
</pre></div>


<p><strong>Python App</strong></p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  host=os.getenv(&#039;DB_HOST&#039;),  # 127.0.0.1
  database=os.getenv(&#039;DB_NAME&#039;),
  user=os.getenv(&#039;DB_USER&#039;),
  password=os.getenv(&#039;DB_PASS&#039;),
)
</pre></div>


<p><strong>Pros</strong></p>



<ul class="wp-block-list">
<li>Very concise configuration in both IaC and application code since there&#8217;s no need to manage client certificates.</li>



<li>Unlike the managed Cloud SQL Proxy volume mount solution, this solution will work even if the Cloud SQL instance has both private and public IPs.</li>
</ul>



<p><strong>Cons</strong></p>



<ul class="wp-block-list">
<li>Need to manage the Cloud SQL Proxy installation and configuration in the Dockerfile.</li>



<li>All the cons described in the managed Cloud SQL Proxy solution above still apply here.</li>
</ul>



<h3 class="wp-block-heading" id="option-d-use-cloud-sq-ls-private-ip-and-client-certificate">Option D: Use Cloud SQL&#8217;s Private IP and Client Certificate</h3>



<p>Instead of relying on external tools/libraries, the application code connects directly to the Cloud SQL instance using the private IP where the client certificate is also provided.</p>



<p><strong>Python App</strong></p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: python; title: ; notranslate">
import os

import pymysql

# ... simplified for brevity

pymysql.connect(
  host=os.getenv(&#039;DB_HOST&#039;), # Cloud SQL&#039;s Private IP
  database=os.getenv(&#039;DB_NAME&#039;),
  user=os.getenv(&#039;DB_USER&#039;),
  password=os.getenv(&#039;DB_PASS&#039;),
  ssl={
    &#039;ca&#039;: os.getenv(&#039;DB_SSL_SERVER_CA&#039;),     # /app/certs/ca/server-ca.pem
    &#039;cert&#039;: os.getenv(&#039;DB_SSL_CLIENT_CERT&#039;), # /app/certs/cert/client-cert.pem
    &#039;key&#039;: os.getenv(&#039;DB_SSL_CLIENT_KEY&#039;),   # /app/certs/key/client-key.pem
    &#039;check_hostname&#039;: False,
  }
)
</pre></div>


<p><strong>Note:</strong> The hostname checking must be disabled because the database host (Cloud SQL&#8217;s Private IP) will never<br>match the common name specified in the certificate. If the hostname checking is not disabled, it will produce the following error:</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p>Error connecting to database: (2003, &#8220;Can&#8217;t connect to MySQL server on &#8217;10.x.x.x&#8217; ([SSL: CERTIFICATE_VERIFY_FAILED]<br>certificate verify failed: IP address mismatch, certificate is not valid for &#8217;10.x.x.x&#8217;. (_ssl.c:1000))&#8221;)</p>
</blockquote>



<p><strong>Pros</strong></p>



<ul class="wp-block-list">
<li>No additional wrappers like Cloud SQL Proxy or Cloud SQL Language Connectors are required.</li>
</ul>



<p><strong>Cons</strong></p>



<ul class="wp-block-list">
<li>Additional configurations and complexity on the IaC and application code are needed to manage and ingest the client certificate securely.</li>
</ul>



<h2 class="wp-block-heading" id="recommendations">Recommendations</h2>



<h3 class="wp-block-heading" id="best-solution-option-a">Best Solution &#8211; Option A</h3>



<p>Use Cloud SQL Language Connectors if your app uses one of the supported programming languages (Python, Java, Go, or Node.js). </p>



<p>It eliminates many unnecessary configurations on the IaC code and application code. In other words, there&#8217;s no need to do the following:</p>



<ul class="wp-block-list">
<li>Create a client certificate for Cloud SQL.</li>



<li>Store server certificate, client certificate, and client key in Secret Manager securely.</li>



<li>Mount files containing the server certificate, client certificate, and client key as volumes in the Cloud Run instance.</li>



<li>Pass certificate information to the application code via environment variables. </li>
</ul>



<p>None of the above is needed. </p>



<p>Less codebase = Less chance of errors.</p>



<h3 class="wp-block-heading" id="distance-next-best-solution-option-d">Distance Next Best Solution &#8211; Option D</h3>



<p>Connect directly using the Cloud SQL instance&#8217;s Private IP and client certificate.</p>



<p>It eliminates the need to rely on additional cloud configuration or external library (i.e., fewer points of failure). </p>



<p>However, it introduces additional configurations in IaC and application code to maintain and ingest the certificate described above.</p>



<h3 class="wp-block-heading" id="least-favorite-solution-option-b-c">Least Favorite Solution &#8211; Option B &amp; C</h3>



<p>While both Cloud SQL Proxy solutions (volume mount in the Cloud Run instance or installing it directly via Dockerfile) don&#8217;t require a client certificate, they come with <a href="https://cloud.google.com/sql/docs/mysql/sql-proxy#production-environment" target="_blank" rel="noopener">additional considerations and complexities when running in production</a>. </p>



<p>Furthermore, the volume mount solution won&#8217;t work if the Cloud SQL instance has both private and public IPs. </p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2024/08/08/cloud-run-connecting-to-cloud-sql-with-private-ip-and-ssl/">Cloud Run: Connecting to Cloud SQL with Private IP and SSL</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://myshittycode.com/2024/08/08/cloud-run-connecting-to-cloud-sql-with-private-ip-and-ssl/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">2642</post-id>	</item>
		<item>
		<title>GitHub Copilot: 10+ Ways to Increase Productivity</title>
		<link>https://myshittycode.com/2023/09/02/github-copilot-10-plus-ways-to-increase-productivity/</link>
					<comments>https://myshittycode.com/2023/09/02/github-copilot-10-plus-ways-to-increase-productivity/#respond</comments>
		
		<dc:creator><![CDATA[Shitty Author]]></dc:creator>
		<pubDate>Sat, 02 Sep 2023 16:08:18 +0000</pubDate>
				<category><![CDATA[AI]]></category>
		<category><![CDATA[Development Tools]]></category>
		<category><![CDATA[Programming Language]]></category>
		<category><![CDATA[GitHub Copilot]]></category>
		<category><![CDATA[VS Code]]></category>
		<guid isPermaLink="false">https://myshittycode.com/?p=2432</guid>

					<description><![CDATA[<p>What is GitHub Copilot GitHub Copilot is dubbed &#8220;an AI pair programmer&#8221; to help us write code faster, using a generative AI model developed through a collaboration between GitHub, OpenAI, and Microsoft Azure AI. These days, there are nothing short of AI-powered tools that can do similar things in this space. For example, Tabnine, Google&#8217;s [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2023/09/02/github-copilot-10-plus-ways-to-increase-productivity/">GitHub Copilot: 10+ Ways to Increase Productivity</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<div class="wp-block-rank-math-toc-block" id="rank-math-toc"><h2>Table of Contents</h2><nav><ul><li><a href="#what-is-git-hub-copilot">What is GitHub Copilot</a></li><li><a href="#prerequisites">Prerequisites</a></li><li><a href="#tips-tricks">Tips &amp; Tricks</a><ul><li><a href="#type-descriptive-code">Type Descriptive Code</a></li><li><a href="#type-comments-first">Type Comments First</a></li><li><a href="#view-multiple-suggestions">View Multiple Suggestions</a></li><li><a href="#generate-test-cases">Generate Test Cases</a></li><li><a href="#internationalization">Internationalization</a></li><li><a href="#generate-realistic-test-data">Generate Realistic Test Data</a></li><li><a href="#learn-id-es-shortcut-keys">Learn IDE&#8217;s Shortcut Keys</a></li><li><a href="#converse-with-q-and-a">Converse with &#8220;q: &#8221; and &#8220;a: &#8221; </a></li><li><a href="#fix-bug">Fix Bug</a></li><li><a href="#list-steps">List Steps</a></li><li><a href="#explain-code">Explain Code</a></li><li><a href="#translate-codebase-to-another-programming-language">Translate Codebase to Another Programming Language</a></li></ul></li><li><a href="#copilot-labs-settings">Copilot Labs Settings</a><ul><li><a href="#show-brushes-lenses">Show Brushes Lenses</a></li><li><a href="#show-test-generation-lenses">Show Test Generation Lenses</a></li></ul></li><li><a href="#what-else">What Else</a></li></ul></nav></div>



<h2 class="wp-block-heading" id="what-is-git-hub-copilot">What is GitHub Copilot</h2>



<p>GitHub Copilot is dubbed &#8220;an AI pair programmer&#8221; to help us write code faster, using a generative AI model developed through a collaboration between GitHub, OpenAI, and Microsoft Azure AI.</p>



<p>These days, there are nothing short of AI-powered tools that can do similar things in this space. For example, <a href="https://www.tabnine.com/" target="_blank" rel="noopener">Tabnine</a>, <a href="https://cloud.google.com/blog/products/application-development/introducing-duet-ai-for-developers" target="_blank" rel="noopener">Google&#8217;s Duet AI</a>, <a href="https://arxiv.org/abs/2305.12050" target="_blank" rel="noopener">Meta&#8217;s CodeCompose</a> (currently used internally at Meta), and <a href="https://about.fb.com/news/2023/08/code-llama-ai-for-coding/#:~:text=Code%20Llama%20is%20a%20code,Llama%20features%20enhanced%20coding%20capabilities." target="_blank" rel="noopener">Code Llama</a>.</p>



<h2 class="wp-block-heading" id="prerequisites">Prerequisites</h2>



<p>You need to <a href="https://github.com/features/copilot#pricing" target="_blank" rel="noopener">obtain a paid license</a> before you can use GitHub Copilot.</p>



<p>Then, install the <strong>GitHub Copilot</strong> extension in your favorite IDE. In this post, I&#8217;m using VS Code because GitHub also created <strong>the GitHub Copilot Labs</strong> extension that is not available in JetBrains IDEs (e.g., IntelliJ, PyCharm, etc.).</p>



<figure class="wp-block-image size-full is-resized"><img decoding="async" src="https://myshittycode.com/wp-content/uploads/2023/09/extensions.png?x45560" alt="github copilot" class="wp-image-2450" style="width:640px" width="640" srcset="https://myshittycode.com/wp-content/uploads/2023/09/extensions.png 1062w, https://myshittycode.com/wp-content/uploads/2023/09/extensions-300x101.png 300w, https://myshittycode.com/wp-content/uploads/2023/09/extensions-1024x343.png 1024w, https://myshittycode.com/wp-content/uploads/2023/09/extensions-768x257.png 768w" sizes="(max-width: 1062px) 100vw, 1062px" /></figure>



<h2 class="wp-block-heading" id="tips-tricks">Tips &amp; Tricks</h2>



<p>Below is a compilation of cool tricks you can do with GitHub Copilot using VS Code.</p>



<h3 class="wp-block-heading" id="type-descriptive-code">Type Descriptive Code</h3>



<p>The easiest way to get started is to continue with your typical coding. If you type descriptive variables or function names, GitHub Copilot can provide better code suggestions. </p>



<p>You can also cycle through the inline suggestions using <a href="https://docs.github.com/en/copilot/configuring-github-copilot/configuring-github-copilot-in-your-environment?tool=vscode" target="_blank" rel="noopener">shortcut keys</a> in your favorite IDE.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/by-descriptive-code.gif?x45560" alt="" class="wp-image-2434"/></figure>



<h3 class="wp-block-heading" id="type-comments-first">Type Comments First</h3>



<p>The challenge of typing very descriptive variables or function names is they get very lengthy, and you risk having your favorite code format tool complaining about the line length. </p>



<p>So, it helps to write them as comments first before you write your code. You can comment above the function you want to create or even at the top of the file.</p>



<p>The most significant advantage of this approach is if your multline comments are clear and precise, it sets proper guardrails for GitHub Copilot to provide better code suggestions based on the context you define (i.e., less hallucination).</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/by-comments.gif?x45560" alt="" class="wp-image-2433"/></figure>



<h3 class="wp-block-heading" id="view-multiple-suggestions">View Multiple Suggestions</h3>



<p>Sometimes, you want to browse all suggestions all at once. </p>



<p>In that case, identify the shortcut key listed under <strong>github.copilot.generate</strong> first.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="594" src="https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel-1024x594.png?x45560" alt="" class="wp-image-2497" srcset="https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel-1024x594.png 1024w, https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel-300x174.png 300w, https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel-768x446.png 768w, https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel-1536x891.png 1536w, https://myshittycode.com/wp-content/uploads/2023/09/open-completions-panel.png 1858w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /></figure>



<p>Then, type the comment and press the shortcut key in the next new line to open up the suggestions in a new tab. You can browse the suggestions and accept the one you like.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1072" height="708" src="https://myshittycode.com/wp-content/uploads/2023/09/show-all-suggestions.gif?x45560" alt="" class="wp-image-2496"/></figure>



<h3 class="wp-block-heading" id="generate-test-cases">Generate Test Cases</h3>



<p>One of the best uses of GitHub Copilot is the ability to create test cases, especially if your function has excellent documentation. </p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/generate-test-single.gif?x45560" alt="" class="wp-image-2439"/></figure>



<p>Here&#8217;s another example where you want to generate multiple assertions.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/generate-test-multiple-assertions.gif?x45560" alt="" class="wp-image-2438"/></figure>



<p>Perhaps you don&#8217;t want to lump multiple assertions into one test case because if one of the assertions fails, the entire test case fails.</p>



<p>In this example, I&#8217;m using the <a href="https://docs.pytest.org/" target="_blank" rel="noopener">pytest</a> framework, which has a decorator that allows the parameterization of the test function&#8217;s argument(s). To refactor the existing test case, GitHub Copilot Labs&#8217; custom brush converts a single test case into 5 test cases.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1130" height="656" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-brush-custom-parameterize-1.gif?x45560" alt="" class="wp-image-2442" style="object-fit:cover"/></figure>



<h3 class="wp-block-heading" id="internationalization">Internationalization</h3>



<p>If you are building an app that users worldwide will use and want to serve the text based on regions, you can easily convert the text to different spoken languages.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/context-recognition.gif?x45560" alt="" class="wp-image-2435"/></figure>



<p>A faster way to perform language conversion is to use GitHub Copilot Labs&#8217; custom brush to convert multiple lines simultaneously.</p>



<h3 class="wp-block-heading" id="generate-realistic-test-data"><figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1106" height="348" class="wp-image-2440" style="object-fit:cover" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-brush-custom-i18n.gif?x45560" alt=""></figure>Generate Realistic Test Data</h3>



<p>Sometimes, you want to generate realistic data for your program or test cases.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/data-generation-1.gif?x45560" alt="" class="wp-image-2436"/></figure>



<p>You can also define the preferred data structure.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/data-generation-2.gif?x45560" alt="" class="wp-image-2437"/></figure>



<p>Perhaps you want to know what world leaders like to eat. As you can see, GitHub Copilot can determine the country leaders and associate the dishes relevant to that country.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/data-generation-3.gif?x45560" alt="" class="wp-image-2456"/></figure>



<h3 class="wp-block-heading" id="learn-id-es-shortcut-keys">Learn IDE&#8217;s Shortcut Keys</h3>



<p>I have used IntelliJ IDEA as my primary IDE for almost 20 years. My brain is fully wired with that particular IDE&#8217;s shortcut keys. To my disappointment, GitHub doesn&#8217;t create the <strong>GitHub Copilot Labs</strong> extension for JetBrains products.</p>



<p>To challenge myself, I want to learn similar shortcut keys for VS Code. While I can search the web for it, the easiest way is to do it directly in the code editor so that it doesn&#8217;t take my attention away during code development.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="200" src="https://myshittycode.com/wp-content/uploads/2023/09/search-engine-vscode-shortcuts.gif?x45560" alt="" class="wp-image-2449"/></figure>



<h3 class="wp-block-heading" id="converse-with-q-and-a">Converse with &#8220;q: &#8221; and &#8220;a: &#8221; </h3>



<p> You can ask GitHub Copilot anything using the <strong>q:</strong> and <strong>a:</strong> construct, even though it&#8217;s unrelated to your codebase.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="200" src="https://myshittycode.com/wp-content/uploads/2023/09/q-a-search-engine.gif?x45560" alt="" class="wp-image-2448"/></figure>



<p>You can use the same construct to ask GitHub Copilot to explain the codebase.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/q-a-explain.gif?x45560" alt="" class="wp-image-2447"/></figure>



<p>You can even argue with GitHub Copilot and defend Chuck Norris&#8217; honor!</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/q-a-converse.gif?x45560" alt="" class="wp-image-2469"/></figure>



<h3 class="wp-block-heading" id="fix-bug">Fix Bug</h3>



<p>You can attempt to fix unobvious bugs. This feature is far from perfect., but it&#8217;s worth trying out.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1168" height="586" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-brush-fix-bug.gif?x45560" alt="" class="wp-image-2443" style="object-fit:cover"/></figure>



<h3 class="wp-block-heading" id="list-steps">List Steps</h3>



<p>Perhaps you inherited a project from a previous team and struggled to understand the written code. In this case, you can list the steps in the codebase.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1246" height="590" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-brush-list-steps.gif?x45560" alt="" class="wp-image-2444" style="object-fit:cover"/></figure>



<h3 class="wp-block-heading" id="explain-code">Explain Code</h3>



<p>Maybe you don&#8217;t want to clutter the existing codebase with unnecessary comments because you don&#8217;t own the codebase. In this case, you can use GitHub Copilot to explain the code instead.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1246" height="590" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-explain.gif?x45560" alt="" class="wp-image-2445" style="object-fit:cover"/></figure>



<h3 class="wp-block-heading" id="translate-codebase-to-another-programming-language">Translate Codebase to Another Programming Language</h3>



<p>You can also translate a block of highlighted code to another programming language.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1246" height="590" src="https://myshittycode.com/wp-content/uploads/2023/09/lab-translate.gif?x45560" alt="" class="wp-image-2446" style="object-fit:cover"/></figure>



<h2 class="wp-block-heading" id="copilot-labs-settings">Copilot Labs Settings</h2>



<p>If you often use the Copilot Labs features, there are 2 settings worth enabling to provide quick access to them.</p>



<figure class="wp-block-image size-large"><img loading="lazy" decoding="async" width="1024" height="394" src="https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings-1024x394.png?x45560" alt="" class="wp-image-2491" srcset="https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings-1024x394.png 1024w, https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings-300x115.png 300w, https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings-768x296.png 768w, https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings-1536x591.png 1536w, https://myshittycode.com/wp-content/uploads/2023/09/copilot-lab-settings.png 1798w" sizes="auto, (max-width: 1024px) 100vw, 1024px" /></figure>



<h3 class="wp-block-heading" id="show-brushes-lenses">Show Brushes Lenses</h3>



<p>When enabled, all brush functions (fix bug, list steps, custom, etc.) are made available when you move your cursor within the function or highlight the code.</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="640" height="480" src="https://myshittycode.com/wp-content/uploads/2023/09/labs-settings-brush.gif?x45560" alt="" class="wp-image-2492"/></figure>



<h3 class="wp-block-heading" id="show-test-generation-lenses">Show Test Generation Lenses</h3>



<p>When enabled, it displays &#8220;Suggest test&#8221; on the function that quickly brings you to the Copilot Labs&#8217; Test Generation section.</p>



<p>Please note the Copilot Labs&#8217; Test Generation currently works with Javascript and Typescript (as of 2023-09-03).</p>



<figure class="wp-block-image size-full"><img loading="lazy" decoding="async" width="1104" height="772" src="https://myshittycode.com/wp-content/uploads/2023/09/labs-settings-test-generation.gif?x45560" alt="" class="wp-image-2493"/></figure>



<h2 class="wp-block-heading" id="what-else">What Else</h2>



<p>Do you know other cool tricks that are not listed here? If so, drop your comments below. I&#8217;m interested to learn from you.</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2023/09/02/github-copilot-10-plus-ways-to-increase-productivity/">GitHub Copilot: 10+ Ways to Increase Productivity</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://myshittycode.com/2023/09/02/github-copilot-10-plus-ways-to-increase-productivity/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">2432</post-id>	</item>
		<item>
		<title>Terraform: Updating State Using &#8220;Moved&#8221; Block</title>
		<link>https://myshittycode.com/2022/04/30/terraform-updating-state-using-moved-block/</link>
					<comments>https://myshittycode.com/2022/04/30/terraform-updating-state-using-moved-block/#respond</comments>
		
		<dc:creator><![CDATA[Shitty Author]]></dc:creator>
		<pubDate>Sat, 30 Apr 2022 16:11:10 +0000</pubDate>
				<category><![CDATA[Infrastructure as Code]]></category>
		<category><![CDATA[Programming Language]]></category>
		<category><![CDATA[IaC]]></category>
		<category><![CDATA[Terraform]]></category>
		<guid isPermaLink="false">https://myshittycode.com/?p=1926</guid>

					<description><![CDATA[<p>This post illustrates how you can rename existing resources or restructure the Terraform codebase without destroying and recreating the resources using moved block introduced in Terraform 1.1. It also explains some limitations using this new construct. PROBLEM: MODIFYING EXISTING RESOURCE NAME Using a simple resource block below as an example… On apply, one resource is [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2022/04/30/terraform-updating-state-using-moved-block/">Terraform: Updating State Using &#8220;Moved&#8221; Block</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p>This post illustrates how you can rename existing resources or restructure the Terraform codebase without destroying and recreating the resources using <strong>moved</strong> block introduced in Terraform 1.1. It also explains some limitations using this new construct.</p>



<h2 class="wp-block-heading">PROBLEM: MODIFYING EXISTING RESOURCE NAME</h2>



<p>Using a simple resource block below as an example…</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;current&quot; {}
</pre></div>


<p>On apply, one resource is created and the state now tracks that resource.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list
random_pet.current
</pre></div>


<p>When changing the resource name from <strong>current</strong> to <strong>new</strong>…</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;new&quot; {}
</pre></div>


<p>The generated plan indicates the resource will be destroyed and recreated, which may cause potential data loss (ex: if the resource is a database) or unintended ripple effects (because the resource ID has changed). </p>



<p>In this case, congratulations! We have become pet killers.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# random_pet.current will be destroyed
# (because random_pet.current is not in configuration)
- resource &quot;random_pet&quot; &quot;current&quot; {
    - id        = &quot;willing-fowl&quot; -&gt; null
    - length    = 2 -&gt; null
    - separator = &quot;-&quot; -&gt; null
  }

# random_pet.new will be created
+ resource &quot;random_pet&quot; &quot;new&quot; {
    + id        = (known after apply)
    + length    = 2
    + separator = &quot;-&quot;
  }

Plan: 1 to add, 0 to change, 1 to destroy.
</pre></div>


<h3 class="wp-block-heading">Solving With Terraform Earlier Than 1.1</h3>



<p>Before Terraform 1.1, the only way to change the resource name without recreating the resource is to use <strong>terraform state mv</strong>.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state mv random_pet.current random_pet.new
Move &quot;random_pet.current&quot; to &quot;random_pet.new&quot;
Successfully moved 1 object(s).
</pre></div>


<p>We can also verify that the resource name has changed in the state.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list
random_pet.new
</pre></div>


<h3 class="wp-block-heading">Solving With Terraform 1.1</h3>



<p>Terraform 1.1 introduced <strong>moved</strong> block where the state change process can be done directly in Terraform source code without using <strong>terraform state mv</strong>.</p>



<p>To pull this off, we changed the resource name from <strong>current</strong> to <strong>new</strong>. Then, a <strong>moved</strong> block is added to facilitate the state change without affecting the provisioned resource.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;new&quot; {}

moved {
  from = random_pet.current
  to   = random_pet.new
}
</pre></div>


<p>Now, we can verify the generated plan to ensure the provisioned resource does not get destroyed and recreated before applying the move.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [7]; title: ; notranslate">
# random_pet.current has moved to random_pet.new
resource &quot;random_pet&quot; &quot;new&quot; {
  id = &quot;willing-fowl&quot;
  # (2 unchanged attributes hidden)
}

Plan: 0 to add, 0 to change, 0 to destroy.
</pre></div>


<p>The state should have the updated resource name.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list
random_pet.new
</pre></div>


<p>So, what if the <strong>moved</strong> block is left in the codebase, and we do another plan or apply?</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform apply
random_pet.new: Refreshing state... &#x5B;id=willing-fowl]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
</pre></div>


<p>As you can see, leaving the <strong>moved</strong> block in the codebase does not affect the state or the resource, and Terraform is smart enough to ignore it. In another word, once the &#8220;move&#8221; has occurred, we can safely remove the <strong>moved</strong> block.</p>



<h2 class="wp-block-heading">PROBLEM: RESTRUCTURING CODEBASE USING &#8220;FOR_EACH&#8221;</h2>



<p>Let&#8217;s try something a little more complicated. </p>



<p>Let&#8217;s assume we have 3 random pets where we call the resource block 3 times. Perhaps, you inherited the old codebase from pre v0.13 days where <strong>for_each</strong> construct was not introduced yet.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;a&quot; {}
resource &quot;random_pet&quot; &quot;b&quot; {}
resource &quot;random_pet&quot; &quot;c&quot; {}
</pre></div>


<p>Here&#8217;s how the state looks like.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list
random_pet.a
random_pet.b
random_pet.c
</pre></div>


<p>Let&#8217;s assume we want to refactor the existing codebase to leverage the <strong>for_each</strong> construct.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;main&quot; {
  for_each = toset(&#x5B;&quot;a&quot;, &quot;b&quot;, &quot;c&quot;])
}
</pre></div>


<h3 class="wp-block-heading">Solving With Terraform Earlier Than 1.1</h3>



<p>With older version of Terraform, we have to perform the following <strong>terraform state mv</strong> 3 times.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state mv random_pet.a &#039;random_pet.main&#x5B;&quot;a&quot;]&#039; 
Move &quot;random_pet.a&quot; to &quot;random_pet.main&#x5B;\&quot;a\&quot;]&quot;
Successfully moved 1 object(s).

$ terraform state mv random_pet.b &#039;random_pet.main&#x5B;&quot;b&quot;]&#039; 
Move &quot;random_pet.b&quot; to &quot;random_pet.main&#x5B;\&quot;b\&quot;]&quot;
Successfully moved 1 object(s).

$ terraform state mv random_pet.c &#039;random_pet.main&#x5B;&quot;c&quot;]&#039; 
Move &quot;random_pet.c&quot; to &quot;random_pet.main&#x5B;\&quot;c\&quot;]&quot;
Successfully moved 1 object(s).
</pre></div>

<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list 
random_pet.main&#x5B;&quot;a&quot;]
random_pet.main&#x5B;&quot;b&quot;]
random_pet.main&#x5B;&quot;c&quot;]
</pre></div>


<h3 class="wp-block-heading">Solving With Terraform 1.1</h3>



<p>To pull this off with Terraform 1.1, we call the <strong>moved</strong> block 3 times.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
resource &quot;random_pet&quot; &quot;main&quot; {
  for_each = toset(&#x5B;&quot;a&quot;, &quot;b&quot;, &quot;c&quot;])
}

moved {
  from = random_pet.a
  to   = random_pet.main&#x5B;&quot;a&quot;]
}

moved {
  from = random_pet.b
  to   = random_pet.main&#x5B;&quot;b&quot;]
}

moved {
  from = random_pet.c
  to   = random_pet.main&#x5B;&quot;c&quot;]
}
</pre></div>


<p>In the generated plan, we want to verify that there are no changes made to the provisioned resources before applying changing the state.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [19]; title: ; notranslate">
# random_pet.a has moved to random_pet.main&#x5B;&quot;a&quot;]
resource &quot;random_pet&quot; &quot;main&quot; {
  id = &quot;poetic-collie&quot;
  # (2 unchanged attributes hidden)
}

# random_pet.b has moved to random_pet.main&#x5B;&quot;b&quot;]
resource &quot;random_pet&quot; &quot;main&quot; {
  id = &quot;glad-lobster&quot;
  # (2 unchanged attributes hidden)
}

# random_pet.c has moved to random_pet.main&#x5B;&quot;c&quot;]
resource &quot;random_pet&quot; &quot;main&quot; {
  id = &quot;suited-fly&quot;
  # (2 unchanged attributes hidden)
}

Plan: 0 to add, 0 to change, 0 to destroy.
</pre></div>


<p>As you can see, we end up with the same outcome.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state list
random_pet.main&#x5B;&quot;a&quot;]
random_pet.main&#x5B;&quot;b&quot;]
random_pet.main&#x5B;&quot;c&quot;]
</pre></div>


<p>The upside is we can still stick with our <strong>terraform apply</strong> to adjust the state without using <strong>terraform state mv</strong>.</p>



<h2 class="wp-block-heading">LIMITATIONS</h2>



<p>It&#8217;s worth pointing out that it&#8217;s not all sunshine and rainbows here. </p>



<p>The <strong>moved</strong> block solves just part of the state manipulation problems, and you may still need to use <strong>terraform state [action]</strong> for other use cases (at least with Terraform 1.1.x).</p>



<h3 class="wp-block-heading">Limitation #1: Removing Resources(s) from State</h3>



<p>In rare cases, you may want Terraform to stop tracking a resource in its state file. Perhaps, you run into a race condition problem when attempting to destroy the resources, which prevents Terraform from completing the process successfully. As a result, all resources are fully destroyed (via &#8220;eventual consistency&#8221;) but unfortunately, a few non-existent resources are still being tracked in the Terraform&#8217;s state file.</p>



<p>In this case, you can&#8217;t remove the resource from the state by assigning an empty string…</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [3]; title: ; notranslate">
moved {
  from = random_pet.current
  to   = &quot;&quot;
}
</pre></div>

<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform apply
╷
│ Error: Invalid expression
│
│   on main.tf line 21, in moved:
│   21:   to   = &quot;&quot;
│
│ A single static variable reference is required: only attribute access and indexing with constant keys. No calculations, function calls, template expressions, etc are
│ allowed here.
</pre></div>


<p>… or to <strong>null</strong>…</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [3]; title: ; notranslate">
moved {
  from = random_pet.current
  to   = null
}
</pre></div>

<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform apply
╷
│ Error: Invalid address
│
│   on main.tf line 21, in moved:
│   21:   to   = null
│
│ Resource specification must include a resource type and name.
╵
</pre></div>


<p>The only way to pull this off is to use <strong>terraform state rm</strong>.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
$ terraform state rm random_pet.current
Removed random_pet.current
Successfully removed 1 resource instance(s).
</pre></div>


<h3 class="wp-block-heading">Limitation #2: Moving Resource(s) Between State Files</h3>



<p>The <strong>moved</strong> block allows us to rename resource or restructure our codebase all within the same state file. However, it is not possible to move resources from one state file to another state file.</p>



<p>For example, let&#8217;s assume we have one Terraform workspace that manages the following 3 resources.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# workspace A

resource &quot;google_folder&quot; &quot;a&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;a&quot;
}

resource &quot;google_folder&quot; &quot;b&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;b&quot;
}

resource &quot;google_folder&quot; &quot;c&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;c&quot;
}
</pre></div>


<p>… and we want to move resource <strong>c</strong> to another workspace.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# Workspace A

resource &quot;google_folder&quot; &quot;a&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;a&quot;
}

resource &quot;google_folder&quot; &quot;b&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;b&quot;
}
</pre></div>

<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# Workspace B

resource &quot;google_folder&quot; &quot;c&quot; {
  parent       = &quot;folders/123456789012&quot;
  display_name = &quot;c&quot;
}
</pre></div>


<p>The only way to pull this off is to identify the resource ID for <strong>c</strong> in workspace A first…</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; highlight: [9]; title: ; notranslate">
# Workspace A

$ terraform state show google_folder.c
# google_folder.c:
resource &quot;google_folder&quot; &quot;c&quot; {
  create_time     = &quot;2022-04-30T15:00:29.114Z&quot;
  display_name    = &quot;c&quot;
  folder_id       = &quot;999999999999&quot;
  id              = &quot;folders/999999999999&quot;
  lifecycle_state = &quot;ACTIVE&quot;
  name            = &quot;folders/999999999999&quot;
  parent          = &quot;folders/123456789012&quot;
}
</pre></div>


<p>Then, remove that resource from workspace A&#8217;s state.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# Workspace A

$ terraform state rm google_folder.c
Removed google_folder.c
Successfully removed 1 resource instance(s).
</pre></div>


<p>Finally, import the resource in workspace B&#8217;s state.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate">
# Workspace B

$ terraform import google_folder.c folders/999999999999
google_folder.c: Importing from ID &quot;folders/999999999999&quot;...
google_folder.c: Import prepared!
  Prepared google_folder for import
google_folder.c: Refreshing state... &#x5B;id=folders/999999999999]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.
</pre></div><p>The post <a rel="nofollow" href="https://myshittycode.com/2022/04/30/terraform-updating-state-using-moved-block/">Terraform: Updating State Using &#8220;Moved&#8221; Block</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://myshittycode.com/2022/04/30/terraform-updating-state-using-moved-block/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">1926</post-id>	</item>
		<item>
		<title>Enabling Python VirtualEnv in JupyterLab</title>
		<link>https://myshittycode.com/2022/03/18/enabling-python-virtualenv-in-jupyterlab/</link>
					<comments>https://myshittycode.com/2022/03/18/enabling-python-virtualenv-in-jupyterlab/#respond</comments>
		
		<dc:creator><![CDATA[Shitty Author]]></dc:creator>
		<pubDate>Fri, 18 Mar 2022 21:26:42 +0000</pubDate>
				<category><![CDATA[Programming Language]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Google Cloud Platform]]></category>
		<category><![CDATA[ipython]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Vertex AI Workbench]]></category>
		<category><![CDATA[virtualenv]]></category>
		<guid isPermaLink="false">https://myshittycode.com/?p=1380</guid>

					<description><![CDATA[<p>This post illustrates how you can enable Python virtualenv in GCP JupyterLab so that you can organize your .ipynb files to use different virtual environments to keep track of Python package dependencies. PROBLEM You are using GCP JupyterLab. You want to adhere to the Python development best practices by not polluting the global environment with [&#8230;]</p>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2022/03/18/enabling-python-virtualenv-in-jupyterlab/">Enabling Python VirtualEnv in JupyterLab</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p>This post illustrates how you can enable Python virtualenv in GCP JupyterLab so that you can organize your <strong>.ipynb</strong> files to use different virtual environments to keep track of Python package dependencies.</p>



<h2 class="wp-block-heading">PROBLEM</h2>



<ul class="wp-block-list"><li>You are using GCP JupyterLab.</li><li>You want to adhere to the Python development best practices by not polluting the global environment with your Python packages so that you can generate a cleaner &#8220;pip freeze&#8221; in the future.</li><li>You want each Notebook file (.ipynb) to have its own environment so that you can run them with different package versions.</li><li>You configured a Python virtual environment, but &#8220;pip install&#8221; from the Notebook file still installs the packages in the global environment.</li><li>You are fed up.</li></ul>



<h2 class="wp-block-heading">SOLUTION</h2>



<h4 class="wp-block-heading">Configuring Virtual Environments</h4>



<p>In the JupyterLab Notebook&#8217;s terminal, create an empty directory to organize all virtual environments.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate">
mkdir virtualenv
</pre></div>


<p>Ensure ipykernel is installed. This is used to create new IPython kernels.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate">
python3 -m pip install ipykernel
</pre></div>


<p>For each new virtual environment, run the following commands to perform these steps:</p>



<ul class="wp-block-list"><li>Get into the base virtual environment directory.</li><li>Define a new virtual environment name. Replace <strong>[NEW_ENV_NAME]</strong> with a new name.</li><li>Create new Python virtual environment.<ul><li>The <strong>&#8211;system-site-packages</strong> option ensures you can still use the &#8220;data-sciencey&#8221; packages that come pre-installed with the GCP JupyterLab Notebook within your new virtual environment. </li></ul></li><li>Jump into the newly created virtual environment.</li><li>Create a new IPython kernel.</li><li>Exit from virtual environment.</li></ul>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate">
cd virtualenv
VENV=&#x5B;NEW_ENV_NAME] # Update this!
python3 -m venv $VENV --system-site-packages
source $VENV/bin/activate
python -m ipykernel install --user --name=$VENV
deactivate
</pre></div>


<h4 class="wp-block-heading">Configuring Notebook File</h4>



<p>Create a new Notebook file (.ipynb).</p>



<p>In <strong>Select Kernel</strong> dialog, select the kernel that you created. In this example, there are 2 new virtual environments (&#8220;smurfs&#8221; and &#8220;thundercats&#8221;).</p>



<div class="wp-block-image size-large"><figure class="aligncenter is-resized"><a href="https://myshittycode.com/wp-content/uploads/2022/03/screen-shot-2022-03-18-at-3.32.19-pm.png?x45560"><img loading="lazy" decoding="async" src="https://myshittycode.com/wp-content/uploads/2022/03/screen-shot-2022-03-18-at-3.32.19-pm.png?x45560" alt="Selecting a kernel in JupyterLab" class="wp-image-1397" width="342" height="342" srcset="https://myshittycode.com/wp-content/uploads/2022/03/screen-shot-2022-03-18-at-3.32.19-pm.png 684w, https://myshittycode.com/wp-content/uploads/2022/03/screen-shot-2022-03-18-at-3.32.19-pm-300x300.png 300w, https://myshittycode.com/wp-content/uploads/2022/03/screen-shot-2022-03-18-at-3.32.19-pm-150x150.png 150w" sizes="auto, (max-width: 342px) 100vw, 342px" /></a></figure></div>



<p>To perform a simple test, install a new package.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate">
%pip install pandas==1.3.0
</pre></div>


<p><strong>IMPORTANT:</strong> You need to use IPython&#8217;s <a href="https://ipython.readthedocs.io/en/stable/interactive/magics.html" target="_blank" rel="noopener">Magics</a> (literally speaking) to ensure the packages are installed in the virtual environment.</p>



<ul class="wp-block-list"><li><strong>%pip</strong> = This uses the pip package manager within the current kernel. Magic!</li><li><strong>! pip</strong> = This uses the pip package manager from the underlying OS. No magic!</li></ul>



<p>From the menu bar, select <strong>Kernel</strong> -&gt; <strong>Restart Kernel and Clear All Outputs</strong>&#8230; . Always restart the kernel when new packages are installed with <strong>%pip</strong>.</p>



<div class="wp-block-image size-large"><figure class="aligncenter is-resized"><a href="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_41_47_pm.png?x45560"><img loading="lazy" decoding="async" src="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_41_47_pm.png?x45560" alt="Kernel -&gt; Restart Kernel and Clear All Outputs in JupyterLab" class="wp-image-1398" width="388" height="332" srcset="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_41_47_pm.png 776w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_41_47_pm-300x257.png 300w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_41_47_pm-768x657.png 768w" sizes="auto, (max-width: 388px) 100vw, 388px" /></a></figure></div>



<p>Inspect the package version. This should show the version you have just installed.</p>


<div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate">
import pandas
print(pandas.__version__)
</pre></div>


<p>To verify this actually works, create another Notebook file pointing to another kernel. In this file, install the same package but with different version.</p>



<figure class="wp-block-image size-large"><a href="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2.png?x45560"><img loading="lazy" decoding="async" width="2394" height="1586" src="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2.png?x45560" alt="Testing Python Virtualenv in JupyterLab" class="wp-image-1394" srcset="https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2.png 2394w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2-300x199.png 300w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2-1024x678.png 1024w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2-768x509.png 768w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2-1536x1018.png 1536w, https://myshittycode.com/wp-content/uploads/2022/03/screen_shot_2022-03-18_at_3_57_05_pm-2-2048x1357.png 2048w" sizes="auto, (max-width: 2394px) 100vw, 2394px" /></a></figure>
<p>The post <a rel="nofollow" href="https://myshittycode.com/2022/03/18/enabling-python-virtualenv-in-jupyterlab/">Enabling Python VirtualEnv in JupyterLab</a> appeared first on <a rel="nofollow" href="https://myshittycode.com">My Shitty Code</a>.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://myshittycode.com/2022/03/18/enabling-python-virtualenv-in-jupyterlab/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">1380</post-id>	</item>
	</channel>
</rss>

<!--
Performance optimized by W3 Total Cache. Learn more: https://www.boldgrid.com/w3-total-cache/?utm_source=w3tc&utm_medium=footer_comment&utm_campaign=free_plugin

Page Caching using Disk: Enhanced 
Lazy Loading (feed)
Database Caching using Disk (Request-wide modification query)

Served from: myshittycode.com @ 2026-02-19 13:01:44 by W3 Total Cache
-->