SSH tunneling (also known as SSH port forwarding) is powerful technique that allows system administrators to create secure encrypted connections between servers, bypass firewalls, and access services securely across networks. Its also known as SSH magics! Whether you’re trying to securely access internal services, create SOCKS proxies, or establish reverse tunnels to overcome network restrictions, SSH tunnels provide flexible solutions for modern networking challenges. This comprehensive guide explores essential SSH tunneling commands that every system administrator should know, complete with practical examples and use cases to enhance your network security toolkit. So let’s get on with some SSH magics, shall we?

It's 2024! Please avoid writing SSH commands like that.

Instead, configure your ~/.ssh/config with LocalForward, RemoteForward, and ProxyJump. This can save you a significant amount of time, especially when using ssh, scp, or rsync to transfer data from a remote server that requires multiple intermediate SSH connections.

SH tunneling and port forwarding can be used to forward TCP traffic over a secure SSH connection from the SSH client to the SSH server, or vice versa. TCP ports or UNIX sockets can be used, but in this post I’ll focus on TCP ports only.

I won’t go into details, but the following post should show enough examples and options to find use in your day-to-day work.

Ever need to work on a remote computer which is behind a typical firewall. Well if there is no inbound connectivity, you can play a bit of leap frog using ssh in order to gain access to that server. Here’s a quick example.
For this example we’ll assume you are sitting at Host A and would like to have access to Internal Host C. Both firewalls are assumed to allow ssh traffic out.
Our goal here is to have the Internal Host C machine start an ssh session to External Host B (which is on the internet).
SSH -R 2200:localhost:22 User@ExternalHostB
This will start an ssh session from Internal Host C to External Host B and ask the ssh daemon to forward all traffic on External Host B’s port 2200 back to Internal Host C’s port 22 over the established ssh session.
Now If I were on External Host B, to get a shell on Internal Host C all I’d have to do is:
SSH -p 2200 User@localhost
Now the USER in the above statement would have to be a valid user on Internal Host C, not External Host B.
To connect from Host A to Internal Host C you can do several things. The easiest is to leap frog.
SSH USER@ExternalHostB
Then
SSH -p 2200 USER@localhost
The first session opens a shell on External Host B. The second opens a shell on Internal Host C by connecting to the reverse shell we started in the first command.
This has been a simple reverse shell ssh post. Stay tuned for more port forwarding fun.

he OED’s earliest listed usage of “log in” in the modern sense of “to open one’s on-line access to a computer” is from the 1963 publication Compatible Time-Sharing System from the MIT Computation Center. [2] I’m not sure if this is truly the first usage of “log in”, but it would make sense if it was, as CTSS, started in 1961, was arguably the first time-sharing operating systems, and so possibly the first system that you needed to log in to. (Before that we only had batch processing systems).

A lot of new hardware security keys (Yubikey, Nitrokey, Titan, etc.) now support FIDO2 (aka U2F aka Webauthn aka Passkey; yes it’s a mess).

So does OpenSSH.

This spells good news for us, because it is far easier to use than previous hardware security types (eg, PKCS#11 and OpenPGP) with ssh.

A key benefit of all this, if done correctly, is that it is actually impossible to access the raw SSH private key, and impossible to use it without the presence of the SK and a human touching it.

Also, ssh agent forwarding becomes safer again, and what’s more, it can be used to let you tap your local key to authenticate even when sshing from remote machine A to remote machine B.

I’m going to call these hardware security keys “SKs” within this article.

I’ve been annoyed at the material out there, which often doesn’t explain what’s happening and suggests insecure practices.

So, I’m going to introduce SKs and FIDO2, show how to use the keys with SSH, explain the role of ssh-agent with all of this, and walk you through all of the steps.

It’s been known for years now that SSH servers can (and should) be hardened by removing weak default algorithms. For example, recent versions of OpenSSH ship with algorithms suspected of being back-doored by the NSA (i.e.: ECDSA with the NIST P-curves), along with other algorithms with sub-128bit security levels.

But did you know that client software can be hardened too?
Why Harden Client Software?

In a world where all servers are properly hardened, there would be no need to re-configure client software. However, server hardening is unfortunately more rare than it should be, and you can only do it for machines you directly control. In that case, it’s very useful to upgrade your client’s defaults so you have assurance that only strong connections will be made.

The new "Simple standalone #SSH Agent for #OpenPGP cards" (https://crates.io/crates/openpgp-card-ssh-agent) is now available as a package for #Arch Linux, by the way :arch: 😏

This agent offers a frictionless UX when using ssh with keys that are stored on OpenPGP card devices: No more ongoing PIN entry required! 🚀

@dvzrv has once again done amazing packaging and documentation work! 🥳 Thank you 😃

See https://wiki.archlinux.org/title/SSH_keys#OpenPGP_card_ssh-agent for details.

some day you try to connect to some new system and get rudely disconnected before you get prompted for a password. The direct answer to what's happening is that you've run into the server's limit on how many different authentication options it will let you try

This week, we discovered that GitHub.com’s RSA SSH private key was briefly exposed in a public GitHub repository

Some authentication means are more secure than others: using a hardware device designed to store a private key without making it possible to ever extract it is more secure than storing the private key in a file. Unfortunately the most secure ones are also more painful to use. Users who have their keys on a device need to carry the device with them, need to type their PIN code every time they initiate a SSH session, etc. This makes it quite difficult to advocate ways more secure than passwords and files for use cases where the security of the access is not the priority.

So the question is: is it possible to store the authentication material more securely than in a file (which can be stolen by some malware), without changing the user experience?

And the answer is: yes, using a TPM!

YubiKeys are hardware security keys that provide One Time Pads (OTP), namely U2F (Universal 2nd Factor) cryptographic tokens through a USB and/or NFC interface. This means you have to explicitly authorize a new SSH session by tapping the YubiKey. The private SSH key, which is normally on your SSD or cloud instance, should be useless to a malicious user who does not have access to the physical YubiKey on which the second private key is stored.

Configuring 2FA (Two Factor Authentication) with YubiKeys on SSH sessions is ideal for bastion hosts, also known as stepping stone servers that connect to your VPC (Virtual Private Cloud).

In simple words, ssh-audit is a tool for ssh server and client auditing. For example, you can use this tool:

Scan for OpenSSH server and client config for security issues
Make sure the correct and recommended algorithm is used by your Linux and Unix boxes
Check for OpenSSH banners and recognize device or software and operating system
Lookup for ssh key exchange, host-keys, encryption, and message authentication code algorithms
Alert developers and sysadmin about config issues, weak/legacy algorithms, and features used by SSH
Historical information from OpenSSH, Dropbear SSH, and libssh
Policy scans to ensure adherence to a hardened/standard configuration

Guardian Agent (now in beta) allows users to securely empower remote hosts to take actions on their behalf, using their SSH credentials. It allows Mosh and SSH users to enable agent forwarding for every connection, even to hosts they may not fully trust.

Guardian Agent is an alternative to traditional ssh-agent forwarding, which can only safely be enabled when connecting to trusted hosts. The traditional ssh-agent protocol doesn't give the agent information about which host is asking to perform a command on the user's behalf, which server that hosts wants to connect to, or which command the host wants to perform:

Include
Include the specified configuration file(s). Multiple pathnames may be specified and each pathname may contain glob(3) wildcards and, for user configurations, shell-like
``~'' references to user home directories. Files without absolute paths are assumed to be in ~/.ssh if included in a user configuration file or /etc/ssh if included from
the system configuration file. Include directive may appear inside a Match or Host block to perform conditional inclusion.

I had my Include statement trailing a Host directive so it was being included into that Host's config.

quicssh is a #QUIC proxy that allows to use QUIC to connect to an #SSH server without needing to patch the client or the server.

build status coverage status
ssh-audit is a tool for ssh server auditing.
Features

SSH1 and SSH2 protocol server support;
grab banner, recognize device or software and operating system, detect compression;
gather key-exchange, host-key, encryption and message authentication code algorithms;
output algorithm information (available since, removed/disabled, unsafe/weak/legacy, etc);
output algorithm recommendations (append or remove based on recognized software version);
output security information (related issues, assigned CVE list, etc);
analyze SSH version compatibility based on algorithm information;
historical information from OpenSSH, Dropbear SSH and libssh;
no dependencies, compatible with Python 2.6+, Python 3.x and PyPy;

This release adds support for FIDO/U2F hardware authenticators to
OpenSSH. U2F/FIDO are open standards for inexpensive two-factor
authentication hardware that are widely used for website
authentication. In OpenSSH FIDO devices are supported by new public
key types "ecdsa-sk" and "ed25519-sk", along with corresponding
certificate types.

ssh-keygen(1) may be used to generate a FIDO token-backed key, after
which they may be used much like any other key type supported by
OpenSSH, so long as the hardware token is attached when the keys are
used. FIDO tokens also generally require the user explicitly authorise
operations by touching or tapping them.

I also use a tool called Storm, which helps you add SSH connections to your SSH config, so you don’t have to remember them all. Y