Squid: Configuring Whitelisted URLs

PROBLEM

To configure a proxy server that only allows whitelisted URLs through.

SOLUTION

Install Squid… in this case, on Ubuntu.

sudo apt install -y squid

Ensure the service is running.

my@shittycode:/etc/squid$ sudo systemctl status squid
● squid.service - Squid Web Proxy Server
Loaded: loaded (/lib/systemd/system/squid.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-04-21 17:20:54 CDT; 3min 3s ago
Docs: man:squid(8)
Process: 9008 ExecStartPre=/usr/sbin/squid --foreground -z (code=exited, status=0/SUCCESS)
Process: 9012 ExecStart=/usr/sbin/squid -sYC (code=exited, status=0/SUCCESS)
Main PID: 9013 (squid)
Tasks: 4 (limit: 44379)
Memory: 16.1M
CGroup: /system.slice/squid.service
├─9013 /usr/sbin/squid -sYC
├─9015 (squid-1) --kid squid-1 -sYC
├─9016 (logfile-daemon) /var/log/squid/access.log
└─9017 (pinger)

Apr 21 17:20:54 shittycode squid[9015]: Max Swap size: 0 KB
Apr 21 17:20:54 shittycode squid[9015]: Using Least Load store dir selection
Apr 21 17:20:54 shittycode squid[9015]: Set Current Directory to /var/spool/squid
Apr 21 17:20:54 shittycode squid[9015]: Finished loading MIME types and icons.
Apr 21 17:20:54 shittycode squid[9015]: HTCP Disabled.
Apr 21 17:20:54 shittycode squid[9015]: Pinger socket opened on FD 14
Apr 21 17:20:54 shittycode squid[9015]: Squid plugin modules loaded: 0
Apr 21 17:20:54 shittycode squid[9015]: Adaptation support is off.
Apr 21 17:20:54 shittycode squid[9015]: Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 12 flags=9
Apr 21 17:20:55 shittycode squid[9015]: storeLateRelease: released 0 objects

Create a file ( /etc/squid/whitelist.txt ) containing the whitelisted URLs. In this example, only one URL is whitelisted.

my@shittycode:/etc/squid$ cat whitelist.txt
www.google.com

To simplify the configuration, backup /etc/squid/squid.conf and create the same file with these minimal configurations.

my@shittycode:/etc/squid$ cat squid.conf

# An ACL named 'whitelist'
acl whitelist dstdomain '/etc/squid/whitelist.txt'

# Allow whitelisted URLs through
http_access allow whitelist

# Block the rest
http_access deny all

# Default port
http_port 3128

Restart the service to pick up the change.

sudo systemctl restart squid

To test the configuration, when hitting a non-whitelisted URL, it should return 403.

my@shittycode:/etc/squid$ curl -x localhost:3128 -I -L yahoo.com
HTTP/1.1 403 Forbidden
Server: squid/4.10
Mime-Version: 1.0
Date: Wed, 21 Apr 2021 22:22:02 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3507
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from shittycode
X-Cache-Lookup: NONE from shittycode:3128
Via: 1.1 shittycode (squid/4.10)
Connection: keep-alive

When hitting a whitelisted URL, it should return 200.

my@shittycode:/etc/squid$ curl -x localhost:3128 -I -L www.google.com
HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Wed, 21 Apr 2021 22:21:03 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Wed, 21 Apr 2021 22:21:03 GMT
Cache-Control: private
Set-Cookie: 1P_JAR=2021-04-21-22; expires=Fri, 21-May-2021 22:21:03 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=214=AAK1Z6cV4cXlOGLIdHrKhiyzW2iBKpkN5-3OXvVrxEGrw-VekbvM1uFMMUAGubhAciT8NcyCVto2fpDPHJXRBECcqJRFTsUDNb3WBUNIgvK0zWpyxz8bl1aSqB22nQhf2fEwfDM9nAkVZyQG8rst054qOfAHO9kDvkrZRWn9HyM; expires=Thu, 21-Oct-2021 22:21:03 GMT; path=/; domain=.google.com; HttpOnly
X-Cache: MISS from shittycode
X-Cache-Lookup: MISS from shittycode:3128
Via: 1.1 shittycode (squid/4.10)
Connection: keep-alive

Nginx: Requesting and Configuring SSL Certificate

INTRO

Step by step instructions for my future self to obtain the SSL certificate and to configure it in Nginx because my fragile little brain cannot retain them at the moment.

INSTRUCTIONS

Generate a private key and store it in a safe place.

openssl genrsa -out myshittycode_com.key 2048  

Generate a certificate signing request (CSR).

openssl req -new -sha256 \
  -key myshittycode_com.key \
  -out myshittycode_com.csr	\
  -subj "/C=US/ST=ShittyState/L=ShittyCity/OU=ShittyUnit/O=ShittyCompany/CN=myshittycode.com"	

Request a SSL certificate from Certificate Authority (ex: Sectigo) using the generated CSR.

If approved, you will receive an email similar to the below.

ssl-certificate

To configure the SSL certificate in Nginx, don’t use Certificate (w/ chain), PEM encoded because it contains the certificates in the following order:-

  • Root CA certificate
  • Intermediate CA certificate
  • Certificate

Nginx wants them in this order:-

  • Certificate
  • Root CA certificate
  • Intermediate CA certificate

To pull this off, download Certificate only, PEM encoded and Root/Intermediate(s) only, PEM encoded.

Then combine them into one file in the proper order.

cat myshittycode_com_cert.cer myshittycode_com_interm.cer > myshittycode_com_bundle.crt

In nginx.conf, add the bundle certificate, private key and server name.

server {
  listen               443;
  ssl                  on;
  ssl_certificate      /etc/pki/tls/certs/myshittycode_com_bundle.crt;
  ssl_certificate_key  /etc/pki/tls/private/myshittycode_com.key;
  server_name          myshittycode.com;

  location / {
    ...
  }
}

Restart Nginx service.

Azure: Deploying WAR File to Tomcat

PROBLEM

Typically, when using ZipDeploy to push a WAR file (ex: my-app.war) to an Azure instance, we need to:-

  • Rename my-app.war to ROOT.war.
  • Place ROOT.war under webapps/.
  • Zip up webapps/.
  • Use ZipDeploy to push the zip file to an Azure instance.

This zip file will automatically be unzipped under site/wwwroot:-

D:\home\site\wwwroot
└── webapps
    └── ROOT.war

Tomcat detects ROOT.war and will try to unpack the WAR file under ROOT/:-

D:\home\site\wwwroot
└── webapps
    ├── ROOT
    │   ├── META-INF
    │   │   └── ...
    │   └── WEB-INF
    │       └── ...
    └── ROOT.war

The problem is sometimes Tomcat is unable to fully unpack ROOT.war because some files may be locked by some running processes. As a result, the web page fails to load and shows 404 error.

SOLUTION

A better and more stable solution is to use WarDeploy to push WAR file to an Azure instance because:-

  • It simplifies the deployment process because we don’t need to create a zip file with webapps/ containing a WAR file named ROOT.war.
  • Instead of relying on Tomcat to unpack the WAR file, WarDeploy will do the unpacking step elsewhere before copying to site/wwwroot/webapps/.

To use WarDeploy, you can use curl command or PowerShell script to do so.

Here’s an example of the PowerShell script:-

Param(
    [string]$filePath,
    [string]$apiUrl,
    [string]$username,
    [string]$password
)

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username, $password)))

Invoke-RestMethod -Uri $apiUrl -Headers @{ Authorization = ("Basic {0}" -f $base64AuthInfo) } -Method POST -InFile $filePath -ContentType "multipart/form-data"

If the PowerShell script above is called deploy.ps1, to push my-app.war to my-app Azure instance:-

deploy.ps1 -filePath D:\my-app.war -apiUrl https://my-app.scm.azurewebsites.net/api/wardeploy -username [USER] -password [PASSWORD]

Depending on the size of the WAR file, you may get “The operation has timed out” error after 100 seconds:-

Invoke-RestMethod : The operation has timed out
At D:\agent\_work\2ea6e947a\my-app\deploy.ps1:18 char:1
+ Invoke-RestMethod -Uri $apiUrl -Headers @{Authorization=("Basic {0}"  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

This will most likely to occur if the WAR file is too big (80MB+).

To fix this, increase the time out duration by adding -TimeoutSec option to Invoke-RestMethod statement:-

Invoke-RestMethod [OTHER_OPTIONS...] -TimeoutSec 300