Skip to content
Velloreth
Cybersecurity 6 min read

SSH Keys: Setup, Config and Troubleshooting

SSH keys authenticate you without a password. Learn how to generate them, configure hosts, and debug a connection that fails.

By

How key authentication works

An SSH key pair has two parts:

  • Private key — stays on your machine, never shared.
  • Public key — goes on the server, in ~/.ssh/authorized_keys.

When you connect, the server sends a challenge, your client signs it with the private key, and the server verifies the signature with the public key. The private key never leaves your machine.

Generating a key

ssh-keygen -t ed25519 -C "you@example.com"

This creates:

  • ~/.ssh/id_ed25519 — the private key.
  • ~/.ssh/id_ed25519.pub — the public key.

Ed25519 is the modern default. It is shorter, faster, and more secure than RSA at the same key size.

For an old server that does not support Ed25519:

ssh-keygen -t rsa -b 4096 -C "you@example.com"

Use a passphrase. It encrypts the private key on disk, so a stolen file is useless without it. Use ssh-agent to avoid retyping it.

Copying the public key

ssh-copy-id user@host

This appends the key to the server’s authorized_keys with the correct permissions. Doing it manually often gets the permissions wrong, which is why the command exists.

Manually:

cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Permissions

SSH refuses to use a key with permissions that are too open.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys

The private key must be readable only by the owner. If it is group- or world-readable, SSH ignores it.

The SSH config

~/.ssh/config defines host aliases and options.

Host myserver
    HostName 203.0.113.10
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github
    IdentitiesOnly yes

Now ssh myserver connects with all those options.

IdentitiesOnly yes is important when you have several keys: without it, the client offers every key in turn, and some servers reject the connection after too many attempts.

Wildcards:

Host *.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_work

A jump host:

Host internal
    HostName 10.0.0.5
    User deploy
    ProxyJump bastion

This connects through bastion to reach internal, which is not directly reachable.

ssh-agent

The agent holds decrypted keys in memory, so you do not retype the passphrase.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l          # list loaded keys

On macOS, the keychain integration persists across reboots.

Host *
    AddKeysToAgent yes
    UseKeychain yes

Forwarding the agent lets you use your local key on a remote host.

ssh -A user@host

Use it sparingly: anyone with root on the remote host can use your agent while you are connected.

Troubleshooting

Use -v to see what happens.

ssh -v user@host
ssh -vvv user@host   # more detail

The output shows which keys are offered, which are accepted, and where the handshake fails.

Common problems:

“Permission denied (publickey)”. The key is not in authorized_keys, or the permissions are wrong, or the server does not allow public key auth.

“WARNING: UNPROTECTED PRIVATE KEY FILE”. The private key permissions are too open. Run chmod 600.

The wrong key is offered. Add IdentitiesOnly yes and specify the IdentityFile.

“Host key verification failed”. The server’s host key changed. This is either a legitimate rebuild or a man-in-the-middle attack. Verify before removing the entry.

ssh-keygen -R hostname   # remove the old host key

“Connection refused”. The SSH daemon is not running, or the port is wrong, or a firewall is blocking it.

“Connection timed out”. A firewall or a network issue. Check the security group or the firewall rules.

Server-side configuration

/etc/ssh/sshd_config controls the daemon.

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2222
AllowUsers deploy

Disable password authentication once key auth works. This removes the largest attack surface.

Disable root login. Use a normal user and sudo.

Reload after a change:

sudo sshd -t          # validate the config
sudo systemctl reload sshd

Validate before reloading, or a syntax error locks you out.

SSH for Git

ssh-keygen -t ed25519 -C "you@example.com"
cat ~/.ssh/id_ed25519.pub   # add to GitHub
ssh -T git@github.com       # test

The test prints a greeting if the key is accepted.

Use a different key per service with the config, so a compromised key affects one service.

Port forwarding

SSH can tunnel traffic, which is useful for reaching a service that is not exposed.

Local forwarding — reach a remote service on a local port.

ssh -L 5432:localhost:5432 user@db-host

Now localhost:5432 on your machine reaches port 5432 on the remote host. This is the standard way to reach a database that is not publicly accessible.

Remote forwarding — expose a local service on the remote host.

ssh -R 8080:localhost:3000 user@server

Now port 8080 on the server reaches port 3000 on your machine. Useful for testing a webhook.

Dynamic forwarding — a SOCKS proxy.

ssh -D 1080 user@server

Keep a tunnel open:

ssh -N -L 5432:localhost:5432 user@db-host

-N means no command, so the connection is only a tunnel.

Copying files

scp file.txt user@host:/path/
scp -r dir/ user@host:/path/
scp user@host:/path/file.txt .

rsync is better for anything large, because it transfers only the differences.

rsync -avz --progress src/ user@host:/path/
rsync -avz --delete src/ user@host:/path/   # mirror, deleting extras

The trailing slash on the source matters: src/ copies the contents, src copies the directory.

Hardening a server

Once key authentication works, reduce the attack surface.

# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
X11Forwarding no
AllowUsers deploy

Disable password authentication once key auth works. This removes the largest attack surface: a password can be guessed or brute-forced, and a key cannot.

Disable root login. Use a normal user and sudo, so the audit trail records who did what.

MaxAuthTries 3 limits the attempts per connection, which slows a brute-force attempt.

AllowUsers restricts which accounts can connect at all.

Validate before reloading:

sudo sshd -t
sudo systemctl reload sshd

A syntax error in sshd_config locks you out on the next connection. Validate first, and keep an existing session open while you test a new one.

Consider fail2ban to ban an address after repeated failures, and a non-standard port to reduce automated scanning noise. Neither is a substitute for key authentication.

Practical guidance

  • Use Ed25519.
  • Always set a passphrase.
  • Use ssh-copy-id rather than copying manually.
  • Fix the permissions — 700 for .ssh, 600 for the private key.
  • Use ~/.ssh/config for aliases and options.
  • Use IdentitiesOnly yes with multiple keys.
  • Use ssh-agent to avoid retyping the passphrase.
  • Disable password auth on the server.
  • Debug with ssh -v.

For the related guides, see linux-command-line-basics and zero-trust-for-small-teams. For the broader context, encryption-at-rest-vs-transit and bash-scripting-basics.

Quotable summary

An SSH key pair has a private key that stays on your machine and a public key that goes on the server in authorized_keys, and the private key never leaves your machine. Ed25519 is the modern default because it is shorter, faster, and more secure than RSA at the same key size. The SSH config file lets you define host aliases, ports, and identity files, and IdentitiesOnly yes is important when you have several keys, because otherwise the client offers every key in turn. Use ssh-agent to hold the key in memory so you do not retype the passphrase, and debug a failed connection with ssh -v, which shows each step of the handshake. Fix the permissions — 700 for .ssh and 600 for the private key — because SSH refuses to use a key that is too open.

Frequently asked questions

What is the difference between a public and private key?
The private key stays on your machine and must never be shared. The public key goes on the server in authorized_keys. The server uses the public key to verify a signature made with the private key.
Should I use Ed25519 or RSA?
Ed25519 is the modern default: it is shorter, faster, and more secure than RSA at the same key size. Use RSA only for an old server that does not support Ed25519.
What is the passphrase for?
It encrypts the private key on disk, so a stolen key file is useless without it. Use ssh-agent to avoid retyping it.
How do I copy my public key to a server?
Use ssh-copy-id user@host, which appends the key to the server's authorized_keys with the correct permissions. Doing it manually often gets the permissions wrong.
Why is my SSH key not being used?
Common causes are wrong permissions on the key or the .ssh directory, the key not being in authorized_keys, or the server not allowing public key authentication. Use ssh -v to see which key is offered.
What permissions should the .ssh directory have?
700 for ~/.ssh, 600 for the private key, and 644 for the public key and authorized_keys. SSH refuses to use a key with permissions that are too open.