Exposing web apps from my local network using Oracle Cloud Free Tier, Caddy, reverse SSH tunnel

Introduction

Why?

• I don't want to buy a new router with port forwarding (and don't want my website to expose my own IP anyway), so I need another way to expose my local server to the public internet.

• I set up my microblogpub using CloudFlare tunnels, but Cloudflare's TOS prohibits excessive media content traffic, and I want to host a photo gallery.

• I also want to know how to do this on a generic VPS, so I have many options to migrate my setup in the future.

• I don't want to pay for a VPS at this time, and the Oracle Cloud Free Tier offering seems fairly generous at this time.

• I want to host my web apps on a local server (Raspberry Pi 4 in my case) to have physical access the server, and to also be able to do operations like uploading to my photo gallery over my local network, and also so that I can use some other cheap, low resource VPS offering in the future for this.

• I'm writing this up because I'm using Oracle Cloud Free Tier, so I ought to document this so I can replicate this setup, should something unexpected happen to my free resources. Maybe it will help guide someone else trying to be thrifty with their hosting. I try to assume not too much knowledge or familiarity with hosting things, because I'm fairly new to this and don't set this up from scratch often.

Requirements

• A domain name

• Oracle Cloud Free Tier Account

  • It can be kind of annoying to register for the free account - it can't be a virtual card number, and I had to make the username match the username of the e-mail being used (although I'm not entirely sure if that was necessary for it to go through)
  • If you can't get the free tier registration working or want to use something else, you can still try to follow along with this guide for the not Oracle Cloud portions

• Home server (computer) running some appropriate Linux distribution, with internet connectivity

  • In my case, a Raspberry Pi 4 using Debian, connected over WiFi

Warnings

• Don't rely on free resources to host something mission critical. I'm using the free resources to proxy/tunnel to my own server hosting stuff like a home page and photo gallery as a hobby project, so the risks of using free resources is OK for my purposes.

• Have a plan to restore and/or migrate things you have running on free resources, should the free resources get terminated for some reason.

• Using a setup like this to proxy/tunnel a media streaming server or something similarly high bandwidth will probably greatly increase your chances of losing free resources.

Part 1: Create and prepare your Oracle Cloud Free Tier VM

Oracle Info on creating your free Compute instance

• See here for what's always free: ttps://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier_topic-Always_Free_Resources.htm#resources

• Oracle's documentation on getting started with creating your first compute instance is here, and should cover the required steps: https://docs.oracle.com/en-us/iaas/Content/GSG/Reference/overviewworkflow.htm#Tutorial__Launching_Your_First_Linux_Instance

My abbreviated(?) instructions on creating your free Compute instance

• Log in, navigate to Compute -> Instances, and click 'Create Instance'

• Placement: Place in whatever availability domain it allows you to for the given resource

• Image and Shape

  • Image: The available Linux images are flavors of Oracle Linux (based on CentOS), CentOS, and Canonical Ubuntu. I am using Ubuntu 22.04
  • Shape: Pick an Always Free-eligible labeled shape series (AMD or Ampere A1). Using Ampere A1 shapes gets you more for free than AMD, but it is an ARM platform architecture, which may be an issue if you intend to use software that does not play nicely on ARM.
  • Shape: Set the OCPUs and Amount of Memory (RAM) to desired values that are always free eligible. I am just using a single Ampere A1 VM with the maximum allowed resources, which is 4 OCPUs and 24GB memory. https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier_topic-Always_Free_Resources.htm#resources

• Networking:

  • Primary Network: Create a new Virtual Cloud Network
  • Subnet: Create a new public subnet
  • Public IPv4 address: Assign a Public IPv4 address

• Add SSH keys: Generate a key pair for me, and save the private key. This allows connecting to your instance via SSH, save this in a secure manner (and don't lose it!)

• Boot Volume: Optionally specify a custom boot volume size. Since I am only planning to manage a single VM, I allocated the entire 200GB available for free to my VM.

• Click Create

Connecting to your newly created cloud server using SSH

• You should have your instance's private key saved from the creation step. Its file extension should be .key

• Move the private key to an appropriate location and change its permissions as shown below. (You will not be able to authenticate to Oracle Cloud VMs without restricting the key's permissions)

  chmod 400  {path_to_your_private_key}
  

• View your instance details. The important information is the Public IP Address and Username.

• Connect to your instance using SSH:

  ssh -i {path_to_your_private_key} {username}@{Public IP}
  

• Once you have this working, you should also securely transfer the private key to the home server that will be tunneled/proxied through the instance, and replicate these steps to ensure your ssh connectivity is working from there for later steps.

Set up DNS records

• You will need the Public IP Address, the same as was used to connect using SSH.

• Log in to wherever your domain's DNS records are managed. This could be your registrar, or could be Cloudflare if you've moved your domain's DNS management to there.

• Set up records like so. My domain is ennui.page, you should replace ennui.page with {your_domain}.{tld}.

  Type	Name	Content
  A	ennui.page	Public IPv4 address of your instance
  CNAME	*	ennui.page
  CNAME	www	ennui.page

Allow HTTP and HTTPS traffic

• You will need to create Ingress Rules on the Virtual Cloud Network tied to your compute instance to allow traffic from port 80 for HTTP, and port 443 for HTTPS. A guide including screenshots of creating the ingress rules is here: https://dev.to/armiedema/opening-up-port-80-and-443-for-oracle-cloud-servers-j35

• You will also need to modify iptables to allow traffic from port 80 and 443. From a SSH session to your instance, do:

  sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT
  sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT
  sudo netfilter-persistent save
  

• You can test that this worked by using curl localhost:80 and curl localhost:443 and seeing that the connections are not refused.

• Since we will use caddy to serve HTTPS, HTTP and port 80 are not really needed, if you wish to keep port 80 closed.

Part 2: Caddy and SSH Tunneling

Caddy

What is Caddy?

• Caddy is a web server with automatic HTTPS. Its use cases are similar to Nginx, such as use as a reverse proxy. (A reverse proxy directs client requests to applications running on certain ports). A nice benefit of using Caddy instead of nginx is its seamless HTTPS automation.

• Documentation: https://caddyserver.com/docs/

Installing Caddy

• Caddy will be installed on your cloud instance, so you should open an SSH session to it.

• Follow the instructions to install Caddy on Ubuntu: https://caddyserver.com/docs/install#debian-ubuntu-raspbian

• Note that installing the package will automatically start and run a caddy systemd service.

Configuring Caddy

• Use your editor of choice to edit your Caddyfile. It can be found at /etc/caddy

• To apply changes made to your Caddyfile, you don't need to stop or restart caddy. You can simply reload it with the command: caddy reload

• To serve a 'Hello world' message at the root of your domain, change your Caddyfile contents to match below, but change ennui.page to be your {your_domain}.{tld}. Once you save the changes to your Caddyfile, do a caddy reload

  ennui.page {
      respond "Hello, world!"
  }
  

• You should now be able to navigate to https://{your_domain}.{tld} and see a "Hello, world!" response, and indication in your browser that the connection is secure.

  • If things aren't working at this stage, you can view logs to troubleshoot Caddy by doing journalctl -u caddy --no-pager | less
  • Likely issues could be DNS records not set up properly, or ingress traffic on port 443 not allowed, so you should review steps above if that is the case.

• Now that we've verified Caddy is serving requests to the domain over HTTPS properly, append the following to your Caddyfile to reverse proxy the a subdomain to some port. For this example, the subdomain is test.ennui.page, and it will reverse proxy requests to test.ennui.page to port 9876. (You should change these values to {whatever_subdomain}.{your_domain}.{tld} you wish to use, and whatever port you wish to use.)

  test.ennui.page {
      reverse_proxy localhost:9876
  }
  

• Going to https://{whatever_subdomain}.{your_domain}.{tld} won't work just yet, since we shouldn't have anything actually running on port 9876.

Reverse SSH tunneling to expose a web application hosted on your home server to your cloud server

What is reverse SSH tunneling?

• I'm still making sense of it myself, honestly. You can try reading this article: https://www.howtogeek.com/428413/what-is-reverse-ssh-tunneling-and-how-to-use-it/

• Basically, it will allow requests to your cloud instance on port 9876 (or whatever port you set your Caddyfile to reverse proxy) to be responded to by a web application running on your home server. On your home server, a reversed SSH session will be active between between the home server's web application port and your cloud server's port 9876.

How to use Reverse SSH Tunneling

• The following will be done from your home server. You should verify you can connect from your home server to your cloud server over SSH before proceeding.

• From the server on your local network, start up a web application server, or find the port of a web application server you already have running, that you would like to serve on https://{whatever_subdomain}.{your_domain}.{tld}

  • If you don't have anything in mind yet, you can start an http server in the current directory on port 9000 using python3 -m http.server 9000. Make sure that no sensitive files are being exposed in the directory you run this from before you proceed.

• Assuming there is a web server running on port 9000 of your home server, and you want it to be accessible via port 9876 of your cloud server, you can do so using the following command:

  ssh -R 9876:127.0.0.1:9000 -i {path_to_your_private_key} {username}@{Public IP}
  

• Note that it will also open an SSH session to your cloud server. You should now be able to reach the content being served from your local server at https://{whatever_subdomain}.{your_domain}.{tld}

• Now, exit from the SSH session via exit. You will no longer be able to reach the local server from your subdomain.

Long Running Reverse SSH Tunnel Using autossh

• At the prior step, notice that the connection between the home server and the cloud server terminated after exiting from the SSH session. There are more flags we could provide to the ssh command to keep it running in the background, but even with those, the SSH tunnel may still die or stop passing traffic.

• autossh is a program that starts an SSH session and monitors it, so it can be automatically restarted should the connection die. https://www.harding.motd.ca/autossh/

• To install it on your home server:

  sudo apt install autossh
  

• To start your tunnel using autossh:

  autossh -M 9236 -N -T -R 9876:127.0.0.1:9000 -i {path_to_your_private_key} {username}@{Public IP}
  

  • -M 9236 assigns port 9236 as the port autossh will use for monitoring. The port doesn't really matter, as long as it's not in use.

• Verify you can reach your web server at port 9876 of your cloud server, and via your subdomain. Congrats, we're done :)