This post illustrates how you can mount a GCS bucket using GCSFuse on your host machine and expose it as a volume to a Docker container.
PROBLEM
You want to volume mount a FUSE-mounted directory to a container, for example:
When attempting to run the container…
docker run -it --rm -v /my-kfc-bucket:/home busybox
… an error occurred:
docker: Error response from daemon: error while creating
mount source path '/my-kfc-bucket': mkdir /my-kfc-bucket:
file exists.
SOLUTION
Unmount the existing FUSE-mounted directory.
sudo umount /my-kfc-bucket
Mount it back with the following option. Because this command with -o allow_other must be executed with sudo privilege, you will need to change the root ownership to yourself (via –uid and –gid) so that you can easily read/write within the directory.
If it is successful, the output should look like this:
Start gcsfuse/0.40.0 (Go version go1.17.6) for app "" using mount point: /my-kfc-bucket
Opening GCS connection...
Mounting file system "gcs-bucket"...
File system has been successfully mounted.
Rerun the docker container.
docker run -it --rm -v /my-kfc-bucket:/home busybox
Now, you can read/write the GCS bucket’s data from the container. In this example, the GCS bucket’s data is located in /home.
Note: If you choose to listen to local port 22, you will most likely to get this error because your local SSH server may already be using it:
bind: Address already in use
channel_setup_fwd_listener: cannot listen to port: 22
Could not request local forwarding.
If this is your first SSH into your VM, you will be prompted to create the SSH key pair. In this case, keep pressing the “Enter” key until it is created.
WARNING: The private SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/shitty_user/.ssh/google_compute_engine.
Your public key has been saved in /Users/shitty_user/.ssh/google_compute_engine.pub.
The key fingerprint is:
SHA256:gmwGL9bfJLi/FYnebZLL0vVBYoZ3XeT/ivSSFCmiRT8 shitty_user@shitty_machine
The key's randomart image is:
+---[RSA 3072]----+
| ..|
| . ..|
| . . o o ..|
| = o .+.E = . .|
| o O +oS= * . .|
| . + +.* +. o .|
| . o.*.oo.o .|
| ..o.+ .+o . |
| ooo ..o |
+----[SHA256]-----+
External IP address was not found; defaulting to using IAP tunneling.
Writing 3 keys to /Users/shitty_user/.ssh/google_compute_known_hosts
Upon a successful port forwarding, the command will hang with the following text:
External IP address was not found; defaulting to using IAP tunneling.
Existing host keys found in /Users/shitty_user/.ssh/google_compute_known_hosts
That is an expected behavior because the SSH tunnel is now established between your local machine and the VM.
Click on Test Connection button and ensure it is successful.
Click OK button.
Under Deployment dialog, select Mappings tab.
Under Mappings tab, click on the folder icon and specify a location to deploy the codebase to.
Click OK button.
Under Remote Host panel, you can now browse and access the files in your VM remotely.
Pushing Codebase from IntelliJ IDEA to VM
To deploy codebase to the VM, right click on the directory, select Deployment > Upload to [VM_NAME].
The codebase should be copied to the location you specified.
Tips: If you makes changes in both your local machine and VM, select Deployment > Sync with Deployed to [VM_NAME]. This allows you to synchronize the changes on both sides.
Running Codebase Remotely
To run the codebase remotely, select Tools > Start SSH Session.
