GCP: Pushing Codebase from IntelliJ IDEA to VM Instance

OBJECTIVE

To push codebase from IntelliJ IDEA (or any JetBrains products) on a local machine to a VM instance in Google Cloud Platform.

To run the codebase remotely.

WHY DO THIS

You want to leverage all the power of a modern IDE on your 4K screen.

You do not want to use remote desktop tools such as VNC or NoMachine due to performance and screen lag problems.

Your team members make fun of your VIM skills.

SOLUTION

Configuring VM Port Forwarding

Log into GCP.

gcloud auth login

Perform port forwarding over SSH using your running VM.

# SYNTAX
gcloud compute ssh VM_NAME \
    --project PROJECT_ID \
    --zone ZONE \
    -- -NL LOCAL_PORT:localhost:REMOTE_PORT

# EXAMPLE
gcloud compute ssh shitty_vm \
    --project shitty_project \
    --zone us-central1-b \
    -- -NL 8888:localhost:22

Note: If you choose to listen to local port 22, you will most likely to get this error because your local SSH server may already be using it:

bind: Address already in use
channel_setup_fwd_listener: cannot listen to port: 22
Could not request local forwarding.

If this is your first SSH into your VM, you will be prompted to create the SSH key pair. In this case, keep pressing the “Enter” key until it is created.

WARNING: The private SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/shitty_user/.ssh/google_compute_engine.
Your public key has been saved in /Users/shitty_user/.ssh/google_compute_engine.pub.
The key fingerprint is:
SHA256:gmwGL9bfJLi/FYnebZLL0vVBYoZ3XeT/ivSSFCmiRT8 shitty_user@shitty_machine
The key's randomart image is:
+---[RSA 3072]----+
|               ..|
|        .      ..|
|  .    . o   o ..|
|   = o .+.E = . .|
|  o O +oS= * .  .|
| . + +.* +. o   .|
|    . o.*.oo.o  .|
|     ..o.+ .+o . |
|      ooo   ..o  |
+----[SHA256]-----+
External IP address was not found; defaulting to using IAP tunneling.
Writing 3 keys to /Users/shitty_user/.ssh/google_compute_known_hosts

Upon a successful port forwarding, the command will hang with the following text:

External IP address was not found; defaulting to using IAP tunneling.
Existing host keys found in /Users/shitty_user/.ssh/google_compute_known_hosts

That is an expected behavior because the SSH tunnel is now established between your local machine and the VM.

Configuring IntelliJ IDEA

In IntelliJ IDEA, select Tools > Deployment > Browser Remote Host

Under Remote Host panel, select button.

Under Add Server dialog:

  • Name: <A Memorable Name… ex: shitty_server>
  • Type: SFTP

Click OK button.

Under Deployment dialog, select button on SSH Configurations.

Under SSH Configurations dialog:

  • Host: localhost
  • Port: 8888 (or the local port you specified)
  • User name: <Your VM’s user name>
  • Authentication type: Key pair
  • Private key file: /<PATH>/.ssh/google_compute_engine

Click on Test Connection button and ensure it is successful.

Click OK button.

Under Deployment dialog, select Mappings tab.

Under Mappings tab, click on the folder icon and specify a location to deploy the codebase to.

Click OK button.

Under Remote Host panel, you can now browse and access the files in your VM remotely.

Pushing Codebase from IntelliJ IDEA to VM

To deploy codebase to the VM, right click on the directory, select Deployment > Upload to [VM_NAME].

The codebase should be copied to the location you specified.

Tips: If you makes changes in both your local machine and VM, select Deployment > Sync with Deployed to [VM_NAME]. This allows you to synchronize the changes on both sides.

Running Codebase Remotely

To run the codebase remotely, select Tools > Start SSH Session.

Select the configured host.

Run the codebase.

Ansible: Handling Multiple Hosts via SSH

PROBLEM

To run Ansible playbook in multiple hosts via SSH.

SOLUTION

Configuring SSH environment

Ensure SSH keypair exists on the current machine (ex: ~/.ssh/id_rsa for private key and ~/.ssh/id_rsa.pub for public key). If you do not have one, create one:

ssh-keygen

Copy the public key (ex: ~/.ssh/id_rsa.pub) to each remote host’s ~/.ssh/authorized_keys. If this file doesn’t exist, create it.

Ensure the current machine’s .ssh/ directory and file have correct permission.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

In the current machine’s /etc/hosts, add all remote hosts.

127.0.0.1      localhost  # current machine
192.168.1.100  donkeykong # remote host 1
192.168.1.200  supermario # remote host 2

In each remote host, enable the remote login and grant yourself the access to this service.

Enabling Remote Login on Mac

Test SSH connection to remote host to ensure they work first before working on Ansible playbook.

ssh user@donkeykong
ssh user@supermario

Creating Ansible Playbook

Create ansible.cfg and define the location of inventory file.

[defaults]
inventory = inventory.yml

Create inventory.yml and define both localhost and remote hosts.

all:
  hosts:
    localhost:
      ansible_connection: local
    donkeykong:
      ansible_user: user
      ansible_ssh_private_key_file: ~/.ssh/id_rsa
    supermario:
      ansible_user: user
      ansible_ssh_private_key_file: ~/.ssh/id_rsa

Run a test to ensure the connection to remote hosts are successful.

ansible all -i inventory.yml -m ping

If successful, the output looks something like this:

localhost | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
donkeykong | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
supermario | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Create main.yml with a very simple task.

- name: all-hosts
  hosts: all
  tasks:
    - name: Capture current dir
      shell: pwd
      register: output

    - name: Display output
      debug: msg='{{ output.stdout }}'

Run the playbook.

ansible-playbook main.yml

If successful, the output looks something like this:

PLAY [all-hosts] *******************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]
ok: [donkeykong]
ok: [supermario]

TASK [Capture current dir] *********************************************************************************************
changed: [localhost]
changed: [donkeykong]
changed: [supermario]

TASK [Display output] **************************************************************************************************
ok: [localhost] => {
    "msg": "/Users/user/myshittycode"
}
ok: [donkeykong] => {
    "msg": "/Users/user"
}
ok: [supermario] => {
    "msg": "/Users/user"
}

PLAY RECAP *************************************************************************************************************
donkeykong                 : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
localhost                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0  
supermario                 : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Controlling the Hosts

Sometimes, you want finer controls on what tasks to be ran in certain hosts.

To run in just one host (ex: donkeykong):

- name: one-host
  hosts: donkeykong
  tasks:
    - ...

To run in all remote hosts except localhost:

- name: all-hosts-except-localhost
  hosts: all:!localhost
  tasks:
    - ...