Imagine being able to do this:

ssh home
ssh laptop
ssh rpi

From anywhere.

It’s possible. All you need is a single static IP addres (let’s call it my-server) and a device you want to access (laptop).

This article builds on ReverseSSHTunnel.

Here’s how you do it.

Step 1: Generate a “relay key”. This allows a device to open a reverse ssh tunnel to the server

Run the following commands:

[laptop] $ cd ~/.ssh
[laptop] $ ssh-keygen -t ed25519

You will be prompted by ssh-keygen, fill in details as such:

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/myuser/.ssh/id_ed25519): id_relay
Enter passphrase (empty for no passphrase): <empty>
Enter same passphrase again: <empty>
Your identification has been saved in id_relay
Your public key has been saved in id_relay.pub
The key fingerprint is:
SHA256:VnlfBGC1ipWNIy1oyk3aC6wsul3ryrVstbWg8hYKj3k myuser@laptop
The key's randomart image is:
+--[ED25519 256]--+
|            ooo..|
|         . + + o |
|        + = B o .|
|     o B . * + . |
|      * S . . .  |
|.  ...oo..       |
| =..=+ +..       |
|o+E=+o. .        |
|oo+B*            |
+----[SHA256]-----+

1. This key allows devices to open a reverse ssh tunnel to the server without having to type in a password.
2. We choose the ed25519 key algorithm because it gives us strong keys.
3. The password must be empty. This is needed so the laptop can automatically connect after boot without user intervention.
4. The id_relay and id_relay.pub will be generated. The id_ is not necessary. I just like to prefix my keys with it to indicate that it is a key.
5. Is that safe? Yes, this key only allows the reverse tunnel setup, but doesn’t allow logins. A second key is needed for that. That second key will require a passphrase.

Step 2: Add a comment to the key denoting its use

[laptop] $ ssh-keygen -c -C "relay-access" -f ~/.ssh/id_relay

This is not a functional part, except for making it clear to you that this key is used for relaying. The key’s filename does not matter, so if you accidentally rename it, the comment will still exist inside the key.

If you run the following command you will see the comment at the end of the public key:

[laptop] $ cat ~/.ssh/id_relay.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN1sVD3Xq/2mqZdzWB3lNmej60y6ppXPR4jt31uwERYM relay-access

Step 3: Create the relay user on the server

If you’re on NixOS

Add this to the users.users attrset:

relay = {
  isNormalUser = true;
  shell = "${pkgs.util-linux.bin}/bin/nologin";
  createHome = false;
};

Setting the shell variable to nologin is important. It prevents a holder of the relay key from logging in as the relay user.

If you’re not on NixOS:

On the server:

[public-server] $ useradd --shell /bin/nologin relay

To create the user.

Option --shell /bin/nologin makes it so one cannot log in to the user at all.

Step 4: Authorize the key on the server

This allows devices to use your server as a relay.

If you’re on NixOS

On NixOS, modify the relay user by copying the contents of ~/.ssh/id_relay.pub into the openssh.authorizedKeys.keys array like so:

relay = {
  isNormalUser = true;
  shell = "${pkgs.util-linux.bin}/bin/nologin";
  createHome = false;
  openssh.authorizedKeys.keys = [
    "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN1sVD3Xq/2mqZdzWB3lNmej60y6ppXPR4jt31uwERYM relay-access"
  ];
};

If you’re not on NixOS

Temporarily enable login shell to /bin/bash: [my-server] $ usermod relay --shell /bin/bash

Copy the just-generated id_relay.pub into the server’s $HOME/.ssh/authorized_keys.

Example: [laptop] $ ssh relay@my-server 'cat - >> ~/.ssh/authorized_keys' < ~/.ssh/id_relay.pub

Disable login shells again on the server: [my-server] $ usermod relay --shell /bin/nologin

Step 5: Test your reverse tunnel

[laptop] $ ssh -o IdentitiesOnly=yes -i $HOME/.ssh/id_relay -NTR 22221:localhost:22 relay@server

This opens an ssh tunnel. The command runs in the foreground. In another terminal, connect to yourself via this tunnel by running:

[laptop] $ ssh -J relay@server localhost -p 22221

You will see the following prompt, do not answer it yet.

The authenticity of host '[localhost]:22221 (<no hostip for proxy command>)' can't be established.
ED25519 key fingerprint is SHA256:Z1iYrjEqo/EYOBP8QXPaDd7ATOYSdwinVz1CpBynUY0
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:3: 192.168.68.115
Are you sure you want to continue connecting (yes/no/[fingerprint])? 

Step 6: Check if the fingerprint is correct on the device

We can see the fingerprint from the ssh-keygen command from step 1. If you lost this information, then you can run the following on the device to retrieve the signatures:

[laptop] $ ssh-keygen -lf <(ssh-keyscan localhost 2>/dev/null)
256 SHA256:Z1iYrjEqo/EYOBP8QXPaDd7ATOYSdwinVz1CpBynUY0 localhost (ED25519)
4096 SHA256:T9XO0NxJHFcRS7PqH6ibqX730Hq0pIjDu3JEgFMuK9o localhost (RSA)

Looks like the ED25519 fingerprint matches. We were not MITM’d by our relay. It is safe to continue.

Step 7: Make the tunnel a service

This step allows the device to open the tunnel as soon as it has a network connection.

It will also retry opening the tunnel if something causes it to close.

On NixOS:

Add the following to the systemd attrset:

services.reverse-tunnel = {
  enable = true;
  description = "Reverse ssh tunnel for accessing this device";
  wantedBy = [ "multi-user.target" ];
  after = [ "network-online.target" ];
  serviceConfig = {
      Type = "simple";
      User = "myuser";
      Environment = [ "PATH=${pkgs.bash}/bin" ];
      ExecStart = "${pkgs.openssh}/bin/ssh -o IdentitiesOnly=yes -i /home/myuser/.ssh/id_relay -NTR 22221:localhost:22 relay@server";
      Restart = "always";
      RestartSec = 30;
    };
};

This will create a system service that’s run as user “myuser”, substitute whatever user you’d like here. There’s no need to run it as root.

• wantedBy: Runs this service when the system is ready for multiple user logins, this is typically after everything is properly up and running.
• after: Ensures we run this service after our network is online. There is no point in running this service if we have no network.
• Restart: Always restart our service if it is down.
• RestartSec: Wait a bit before restarting.

If you’re not on NixOS

You’ll need to create a service file and install it into systemd. You can copy the values from the NixOS setup.

Step 8: Add a clause to ~/.ssh/config to easily ssh device

Add the following to ~/.ssh/config on the laptop system, and any other system that wants to use ssh laptop.

Host laptop
    User myuser
    Hostname localhost
    Port 22221
    ProxyJump relay@my-server

This will allow you to run ssh laptop to connect to the reverse-tunneled system.

Step 8b: Logins without ssh-agent

If you are using hosts without an SSH-agent, you can also define the relay and which key it should use.

Host relay
    User relay
    Hostname my-server
    ExitOnForwardFailure yes
    UserKnownHostsFile ~/.ssh/known/relayers
    IdentityFile ~/.ssh/id_relay_access
    IdentitiesOnly yes

Then for the laptop host we just use “relay” directly

Host laptop
    User myuser
    Hostname localhost
    Port 22221
    ProxyJump relay

Step 9: Caveats

For each device you add to the same server, you’ll need to use a different port. We used 22221, so the next one must be different from that.