KB
Access & VPN

SSH Key Management and Secure Remote Access

6 min read1190 words20 code blocks

At a glance#

  • Purpose: Generate, deploy, rotate and revoke SSH keys, and configure secure remote access.
  • Applies to: OpenSSH on any Linux distribution; clients on Linux, macOS and Windows.
  • Risk: High — SSH misconfiguration can lock every administrator out of a server.
  • Time: 30 minutes.

Overview#

SSH keys replace passwords with a key pair: a private key that never leaves the client, and a public key installed on each server. They are both more secure and more convenient than passwords, and they are what makes automation possible.

The part that gets neglected is the lifecycle. Keys are generated and deployed, then never rotated, never inventoried, and never removed when someone leaves. A departed administrator's key sitting in authorized_keys is a live credential.

Warning: Before disabling password authentication, confirm your key works from a new session. Every server has a story about someone who set PasswordAuthentication no, closed their terminal, and needed console access to get back in.

Generating a key pair#

1. Create the key#

bash
ssh-keygen -t ed25519 -C "jsmith@company - workstation" -f ~/.ssh/id_ed25519
TypeUse
ed25519Preferred. Short, fast, strong. Supported by OpenSSH 6.5+ (2014).
rsa -b 4096Only when the target is too old for ed25519.
Warning: Set a passphrase when prompted. An unprotected private key is a plaintext credential — anyone who copies the file has your access. Use ssh-agent so you only type it once per session.

The -C comment matters more than it looks. It is what appears in authorized_keys, and it is how you identify whose key to remove two years later.

2. Protect the private key#

bash
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
Warning: SSH refuses to use a private key with permissions looser than 600. It also refuses authorized_keys if the home directory or .ssh is group-writable — a common cause of "key rejected for no reason".

3. Load into the agent#

bash
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l          # list loaded keys

On Windows, use the built-in OpenSSH agent:

powershell
Get-Service ssh-agent | Set-Service -StartupType Automatic
Start-Service ssh-agent
ssh-add $env:USERPROFILE\.ssh\id_ed25519

Deploying a public key#

Preferred method#

bash
ssh-copy-id -i ~/.ssh/id_ed25519.pub jsmith@server.example.com

This appends the key and fixes permissions automatically.

Manual method#

When ssh-copy-id is unavailable:

bash
cat ~/.ssh/id_ed25519.pub | ssh jsmith@server.example.com \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
Warning: Use >> not >. A single > overwrites authorized_keys and removes every other key on that account — including your colleagues'.

Verify before going further#

bash
ssh -i ~/.ssh/id_ed25519 jsmith@server.example.com

It should log in without a password prompt. Only once this works should you consider disabling password authentication.

Client configuration#

~/.ssh/config removes repetitive typing and makes access consistent:

text
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    HashKnownHosts yes

Host web01
    HostName 192.168.10.21
    User anup
    IdentityFile ~/.ssh/id_ed25519

Host db01
    HostName 192.168.10.31
    User anup
    IdentityFile ~/.ssh/id_ed25519
    Port 2222

# reach an internal host through a bastion
Host internal-app
    HostName 10.0.5.14
    User anup
    ProxyJump bastion.example.com
bash
chmod 600 ~/.ssh/config
ssh web01

ProxyJump is the correct way to reach hosts behind a bastion. It tunnels through without ever placing your private key on the intermediate host — unlike agent forwarding, which exposes your agent to anyone with root on the bastion.

Server hardening#

bash
sudo nano /etc/ssh/sshd_config
text
# authentication
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
AuthenticationMethods publickey

# session
ClientAliveInterval 300
ClientAliveCountMax 2
LoginGraceTime 30

# restrict who may connect
AllowGroups sshusers

# disable unused features
X11Forwarding no
AllowAgentForwarding no

Always validate before restarting:

bash
sudo sshd -t

Then, with a second session still open:

bash
sudo systemctl restart sshd

Open a new session to confirm before closing anything.

Note: AllowGroups sshusers means only members of that group can connect at all. Create it and add users before enabling, or nobody will be able to log in: ``bash sudo groupadd sshusers sudo usermod -aG sshusers anup ``

Key inventory and rotation#

Audit which keys exist#

Run across the estate periodically:

bash
# every authorized_keys on the server, with owner
sudo find /home /root -name authorized_keys -exec sh -c \
  'echo "=== $1 ==="; cat "$1"' _ {} \;

The comment field on each line identifies the owner. Keys with no comment are exactly the ones to investigate.

Show fingerprints:

bash
ssh-keygen -lf ~/.ssh/authorized_keys

Rotate a key#

  1. Generate a new pair with a new comment.
  2. Deploy the new public key alongside the old one.
  3. Confirm the new key works from a fresh session.
  4. Remove the old key line from authorized_keys.
  5. Delete the old private key from the client.

Rotate annually, and immediately if a client machine is lost or a passphrase may be known.

Revoke access#

When someone leaves, remove their key from every server:

bash
sudo sed -i '/jsmith@company/d' /home/*/.ssh/authorized_keys /root/.ssh/authorized_keys

Verify it is gone:

bash
sudo grep -r "jsmith@company" /home/*/.ssh/authorized_keys /root/.ssh/authorized_keys
Warning: Locking the user account with usermod -L does not block key-based login. Removing the key is the step that actually revokes access. See Linux User and Group Management.

Restricting what a key can do#

For automation accounts, constrain the key rather than granting a shell:

text
# in authorized_keys, all on one line before the key
command="/usr/local/bin/backup-receive.sh",no-port-forwarding,no-agent-forwarding,no-pty ssh-ed25519 AAAA... backup@jobserver

That key can only run the named script, cannot open a shell, and cannot forward ports. This is how a backup or deployment key should always be deployed.

Verification#

bash
# server config valid
sudo sshd -t

# effective settings, including defaults
sudo sshd -T | grep -iE "permitrootlogin|passwordauth|pubkeyauth|allowgroups"

# key-based login works
ssh -i ~/.ssh/id_ed25519 user@server

# password login is refused
ssh -o PubkeyAuthentication=no user@server     # expect: Permission denied

# root login is refused
ssh root@server                                # expect: Permission denied

Check what is actually happening if a login fails:

bash
ssh -vvv user@server 2>&1 | grep -iE "offering|authentications that can continue|denied"
sudo journalctl -u sshd -n 50

Troubleshooting#

SymptomCause and fix
Key ignored, still prompts for passwordPermissions. ~ must not be group-writable; .ssh 700; authorized_keys 600.
Permission denied (publickey)Key not in authorized_keys, or wrong username. Check with ssh -vvv.
Works for one user, not anotherKey deployed to the wrong account's authorized_keys.
Agent admitted failure to signKey not loaded. Run ssh-add ~/.ssh/id_ed25519.
Locked out after disabling passwordsConsole access required. Set PasswordAuthentication yes, restart sshd, fix the key.
sshd will not startSyntax error. sudo sshd -t names the line.
REMOTE HOST IDENTIFICATION HAS CHANGEDServer rebuilt or key changed. Confirm it is expected, then ssh-keygen -R hostname.
Connection drops when idleAdd ServerAliveInterval 60 client-side.
Login very slowReverse DNS lookup failing. Set UseDNS no in sshd_config.
Key works but user cannot sudoSeparate issue — check group membership.