SSH Keys: Setup, Config and Troubleshooting
SSH keys authenticate you without a password. Learn how to generate them, configure hosts, and debug a connection that fails.
By Velloreth
How key authentication works
An SSH key pair has two parts:
- Private key — stays on your machine, never shared.
- Public key — goes on the server, in
~/.ssh/authorized_keys.
When you connect, the server sends a challenge, your client signs it with the private key, and the server verifies the signature with the public key. The private key never leaves your machine.
Generating a key
ssh-keygen -t ed25519 -C "you@example.com"
This creates:
~/.ssh/id_ed25519— the private key.~/.ssh/id_ed25519.pub— the public key.
Ed25519 is the modern default. It is shorter, faster, and more secure than RSA at the same key size.
For an old server that does not support Ed25519:
ssh-keygen -t rsa -b 4096 -C "you@example.com"
Use a passphrase. It encrypts the private key on disk, so a stolen file is useless without it. Use ssh-agent to avoid retyping it.
Copying the public key
ssh-copy-id user@host
This appends the key to the server’s authorized_keys with the correct permissions. Doing it manually often gets the permissions wrong, which is why the command exists.
Manually:
cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
Permissions
SSH refuses to use a key with permissions that are too open.
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys
The private key must be readable only by the owner. If it is group- or world-readable, SSH ignores it.
The SSH config
~/.ssh/config defines host aliases and options.
Host myserver
HostName 203.0.113.10
User deploy
Port 2222
IdentityFile ~/.ssh/id_ed25519
Host github.com
User git
IdentityFile ~/.ssh/id_ed25519_github
IdentitiesOnly yes
Now ssh myserver connects with all those options.
IdentitiesOnly yes is important when you have several keys: without it, the client offers every key in turn, and some servers reject the connection after too many attempts.
Wildcards:
Host *.example.com
User deploy
IdentityFile ~/.ssh/id_ed25519_work
A jump host:
Host internal
HostName 10.0.0.5
User deploy
ProxyJump bastion
This connects through bastion to reach internal, which is not directly reachable.
ssh-agent
The agent holds decrypted keys in memory, so you do not retype the passphrase.
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l # list loaded keys
On macOS, the keychain integration persists across reboots.
Host *
AddKeysToAgent yes
UseKeychain yes
Forwarding the agent lets you use your local key on a remote host.
ssh -A user@host
Use it sparingly: anyone with root on the remote host can use your agent while you are connected.
Troubleshooting
Use -v to see what happens.
ssh -v user@host
ssh -vvv user@host # more detail
The output shows which keys are offered, which are accepted, and where the handshake fails.
Common problems:
“Permission denied (publickey)”. The key is not in authorized_keys, or the permissions are wrong, or the server does not allow public key auth.
“WARNING: UNPROTECTED PRIVATE KEY FILE”. The private key permissions are too open. Run chmod 600.
The wrong key is offered. Add IdentitiesOnly yes and specify the IdentityFile.
“Host key verification failed”. The server’s host key changed. This is either a legitimate rebuild or a man-in-the-middle attack. Verify before removing the entry.
ssh-keygen -R hostname # remove the old host key
“Connection refused”. The SSH daemon is not running, or the port is wrong, or a firewall is blocking it.
“Connection timed out”. A firewall or a network issue. Check the security group or the firewall rules.
Server-side configuration
/etc/ssh/sshd_config controls the daemon.
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
Port 2222
AllowUsers deploy
Disable password authentication once key auth works. This removes the largest attack surface.
Disable root login. Use a normal user and sudo.
Reload after a change:
sudo sshd -t # validate the config
sudo systemctl reload sshd
Validate before reloading, or a syntax error locks you out.
SSH for Git
ssh-keygen -t ed25519 -C "you@example.com"
cat ~/.ssh/id_ed25519.pub # add to GitHub
ssh -T git@github.com # test
The test prints a greeting if the key is accepted.
Use a different key per service with the config, so a compromised key affects one service.
Port forwarding
SSH can tunnel traffic, which is useful for reaching a service that is not exposed.
Local forwarding — reach a remote service on a local port.
ssh -L 5432:localhost:5432 user@db-host
Now localhost:5432 on your machine reaches port 5432 on the remote host. This is the standard way to reach a database that is not publicly accessible.
Remote forwarding — expose a local service on the remote host.
ssh -R 8080:localhost:3000 user@server
Now port 8080 on the server reaches port 3000 on your machine. Useful for testing a webhook.
Dynamic forwarding — a SOCKS proxy.
ssh -D 1080 user@server
Keep a tunnel open:
ssh -N -L 5432:localhost:5432 user@db-host
-N means no command, so the connection is only a tunnel.
Copying files
scp file.txt user@host:/path/
scp -r dir/ user@host:/path/
scp user@host:/path/file.txt .
rsync is better for anything large, because it transfers only the differences.
rsync -avz --progress src/ user@host:/path/
rsync -avz --delete src/ user@host:/path/ # mirror, deleting extras
The trailing slash on the source matters: src/ copies the contents, src copies the directory.
Hardening a server
Once key authentication works, reduce the attack surface.
# /etc/ssh/sshd_config
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
MaxAuthTries 3
X11Forwarding no
AllowUsers deploy
Disable password authentication once key auth works. This removes the largest attack surface: a password can be guessed or brute-forced, and a key cannot.
Disable root login. Use a normal user and sudo, so the audit trail records who did what.
MaxAuthTries 3 limits the attempts per connection, which slows a brute-force attempt.
AllowUsers restricts which accounts can connect at all.
Validate before reloading:
sudo sshd -t
sudo systemctl reload sshd
A syntax error in sshd_config locks you out on the next connection. Validate first, and keep an existing session open while you test a new one.
Consider fail2ban to ban an address after repeated failures, and a non-standard port to reduce automated scanning noise. Neither is a substitute for key authentication.
Practical guidance
- Use Ed25519.
- Always set a passphrase.
- Use
ssh-copy-idrather than copying manually. - Fix the permissions — 700 for
.ssh, 600 for the private key. - Use
~/.ssh/configfor aliases and options. - Use
IdentitiesOnly yeswith multiple keys. - Use
ssh-agentto avoid retyping the passphrase. - Disable password auth on the server.
- Debug with
ssh -v.
For the related guides, see linux-command-line-basics and zero-trust-for-small-teams. For the broader context, encryption-at-rest-vs-transit and bash-scripting-basics.
Quotable summary
An SSH key pair has a private key that stays on your machine and a public key that goes on the server in authorized_keys, and the private key never leaves your machine. Ed25519 is the modern default because it is shorter, faster, and more secure than RSA at the same key size. The SSH config file lets you define host aliases, ports, and identity files, and IdentitiesOnly yes is important when you have several keys, because otherwise the client offers every key in turn. Use ssh-agent to hold the key in memory so you do not retype the passphrase, and debug a failed connection with ssh -v, which shows each step of the handshake. Fix the permissions — 700 for .ssh and 600 for the private key — because SSH refuses to use a key that is too open.