Securing SSH: key login, blocking root login, disabling password authentication

A freshly installed server appears on the scanners' lists within minutes of becoming reachable for the first time. What arrives is almost always the same thing: automated guessing of usernames and passwords on port 22. If you turn off password authentication and allow keys only, you take away the basis for that entire class of attack. Not weakened, but removed completely.

Most people know the basic steps. What the usual guides leave out is everything that comes afterwards: why PasswordAuthentication no regularly has no effect on cloud images, why a systemctl reload ssh on a server with an active ssh.socket no longer does what you expect, and how to prove rather than hope that your change took hold. That is what this article is about. Everything below applies to Debian 13, Debian 12, Ubuntu 24.04 and Ubuntu 22.04.

The rule that saves everything: two sessions

Before you change anything in the SSH configuration, open a second terminal window and log in to the server there as well. Keep this second session open until you have successfully tested the new configuration with a third, completely new connection.

The reason is technical: restarting the SSH service does not terminate existing sessions. Running connections are served by child processes that were forked off earlier, and they survive a restart of the parent process. So you only notice a broken configuration on the next connection attempt, and by then the old session is your only way back. Anyone who closes it "just to log in cleanly one more time" regrets it with some regularity.

Check as well which OpenSSH version you are dealing with, because several details depend on it:

ssh -V

If the server component is missing altogether, for example in a minimal image, install it:

sudo apt update
sudo apt install -y openssh-server

Generating a key: ed25519 on Linux, macOS and Windows

Use ed25519. It is short, fast to verify, has a large security margin, and every OpenSSH version covered here supports it. You only need RSA for legacy systems that accept nothing else, and then with at least 4096 bits.

Linux and macOS

ssh-keygen -t ed25519 -a 100 -C "hani@notebook" -f ~/.ssh/id_ed25519

-a 100 raises the number of KDF rounds for the passphrase and makes offline attacks on the key file considerably more expensive. -C sets a comment that later shows up in authorized_keys and tells you which key came from which device. Set a passphrase. A key without one is just a file that anybody who sits at your machine for a moment can walk away with.

So that you do not have to type the passphrase on every connection, the agent takes care of caching it:

eval "$(ssh-agent -s)" && ssh-add ~/.ssh/id_ed25519

On macOS you put the passphrase into the keychain instead. The former -K switch has been called --apple-use-keychain since macOS 12:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

For that to survive a reboot, this belongs in ~/.ssh/config:

Host *
    UseKeychain yes
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_ed25519

Windows

Windows 10 from version 1809, Windows 11 and Windows Server from 2019 ship with the OpenSSH client. In PowerShell:

ssh -V
ssh-keygen -t ed25519 -a 100 -C "hani@windows"

The key ends up in C:\Users\YourName\.ssh\. If the client is missing, install it as a Windows capability:

Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

On Windows the agent is a system service, and it is disabled in the shipped state:

Get-Service ssh-agent | Set-Service -StartupType Automatic
Start-Service ssh-agent
ssh-add $env:USERPROFILE\.ssh\id_ed25519

Getting the public key onto the server

Only the file ending in .pub belongs on the server. The file without that ending is the private key and never leaves your own machine.

The convenient way on Linux

ssh-copy-id -i ~/.ssh/id_ed25519.pub root@203.0.113.10

ssh-copy-id creates ~/.ssh, sets the permissions correctly, appends the key to authorized_keys and checks whether it is already there. On macOS the tool is not included in every version. Check quickly with command -v ssh-copy-id and switch to the manual route if it is missing.

The Windows route without ssh-copy-id

Microsoft's OpenSSH client does not include ssh-copy-id. The original is a shell script and was never ported. You get the same result with a pipe:

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh root@203.0.113.10 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

There is a detail lurking here that costs many hours: when handing data to an external program, PowerShell appends line endings in Windows format. An invisible carriage return then sits at the end of the line in authorized_keys. As long as you commented the key with -C, that character lands in the comment field and does no harm. Without a comment it sticks to the Base64 block, and the login fails without any usable message. So: always set a comment, and when in doubt, clean up once on the server.

sed -i 's/\r$//' ~/.ssh/authorized_keys

The manual way that works everywhere

mkdir -p ~/.ssh && chmod 700 ~/.ssh && touch ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys

Then append the contents of the .pub file as one single line. If the file is already on the server, because you uploaded it for example, you can do it directly:

cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys

Afterwards, check what actually ended up in the file:

ssh-keygen -lf ~/.ssh/authorized_keys

The output lists the fingerprint and comment of every valid entry. Anything that does not show up here is line wrapped or damaged. Compare the fingerprint with the one of your local key:

ssh-keygen -l -f ~/.ssh/id_ed25519.pub

If the two match, the transfer went cleanly. Now test whether key authentication works at all, before you switch anything off. Only once a new connection goes through without a password prompt do you carry on.

The trap: /etc/ssh/sshd_config.d and cloud-init

This is the point where most guides stop, and the most common reason for the sentence "I turned it off and it still works".

Since Debian 11 and Ubuntu 22.04 there is a line right at the top of /etc/ssh/sshd_config that reads in an entire directory. Have a look yourself at which position it sits:

grep -n Include /etc/ssh/sshd_config

On all four systems covered here, the Include line sits at the beginning, not at the end. And now comes the OpenSSH quirk that you will find in almost no other configuration language: the first value found wins, not the last. So whatever sits in a file under /etc/ssh/sshd_config.d/ is read before the main file and beats every later line in sshd_config.

Cloud-init uses exactly this directory. On first boot it writes /etc/ssh/sshd_config.d/50-cloud-init.conf, and depending on the deployment that file contains PasswordAuthentication yes. After that you can put PasswordAuthentication no into sshd_config as often as you like: password authentication stays open. So get an overview first:

ls -la /etc/ssh/sshd_config.d/
sudo grep -riE 'passwordauthentication|permitrootlogin|kbdinteractive' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/

From this follows the actual recommendation: do not touch sshd_config in the first place. Create your own file instead, with a name that sorts alphabetically before anything the automation drops there. The files are read in sorted order, 01- comes before 50-, and because the first value wins, cloud-init can rewrite its file as often as it likes without undoing your hardening. That is the difference to the widespread advice to create a 99-hardening.conf: that one loses against cloud-init, and it loses silently. The same mechanism can turn against you, though: a file with a lower number, 00-cloud.conf for instance, beats your 01-, because OpenSSH takes the value it reads first. So it is worth looking into the directory again after hardening.

Writing the hardening, with a timed rollback as a fallback

Build the safety net first. The following command removes your new file automatically in ten minutes and restarts SSH, unless you have cancelled it by then:

sudo systemd-run --on-active=10min --unit=ssh-rollback /bin/sh -c 'rm -f /etc/ssh/sshd_config.d/01-hardening.conf; systemctl try-restart ssh.socket ssh.service'

Now the actual configuration:

sudo tee /etc/ssh/sshd_config.d/01-hardening.conf >/dev/null <<'EOF'
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin prohibit-password
PermitEmptyPasswords no
MaxAuthTries 3
EOF

sudo chmod 644 /etc/ssh/sshd_config.d/01-hardening.conf

Two lines deserve an explanation. KbdInteractiveAuthentication no is not decoration: if it stays at yes, PAM can still offer the password prompt through the detour of keyboard interactive input, even though PasswordAuthentication is set to no. That is exactly why there are servers that keep asking for a password despite password authentication being switched off.

And PermitRootLogin prohibit-password instead of no: this keeps root access by key available while ruling out passwords for root. Once you have set up your own user with sudo and verifiably tested that user's key login, switch to PermitRootLogin no. Not before.

What you should not adopt, even though older guides mention it: ChallengeResponseAuthentication. Since OpenSSH 8.7 the option is nothing more than a deprecated alias for KbdInteractiveAuthentication, and on current versions it produces a deprecation message in the log. Leave it out.

Syntax check before you activate anything, without exception:

sudo sshd -t

No output means the file is syntactically fine. It does not mean it does what you want. That is the job of sudo sshd -T in a moment.

If the test reports Missing privilege separation directory: /run/sshd instead, then sshd has not run a single time since the system booted, because that directory is only created by ssh.service. This mainly hits Ubuntu 24.04 with socket activation and can be cleared up with sudo mkdir -p /run/sshd or sudo systemctl start ssh.service. On a server where you are currently logged in over SSH it will not occur anyway.

Activating: ssh.service, ssh.socket and the differences per distribution

This is where the four systems part ways, and the old habit of systemctl reload ssh is the wrong answer everywhere ssh.socket holds the port.

Ubuntu has relied on socket activation since 22.10, and Ubuntu 24.04 ships it by default: there ssh.socket is enabled and ssh.service is disabled. Debian does it exactly the other way round out of the box. Debian 13 and Debian 12 ship with ssh.service enabled and ssh.socket disabled, contrary to the widespread assumption that Debian 13 switches to the socket on a fresh installation. With socket activation, systemd itself listens on port 22 and only starts a fresh sshd once an incoming connection arrives. That has a pleasant side effect: changes to sshd_config take hold on the next connection anyway, because every connection process reads the configuration again. You only need a reload for settings that concern the listener itself, meaning Port and ListenAddress.

So do not guess, ask the system which unit is doing the job on your machine:

systemctl is-enabled ssh.socket ssh.service

And then a command that is correct on all four systems:

sudo sshd -t && sudo systemctl try-restart ssh.socket ssh.service

try-restart only restarts a unit if it is actually active, and leaves the other one untouched. That way you do not have to guess. Both parts need root: sshd lives in /usr/sbin and is not in the PATH of a normal user on Debian, and try-restart talks to systemd anyway. Without sudo, the call ends with sshd: command not found or sshd: no hostkeys available, depending on the system, because the host keys under /etc/ssh are readable by root only.

Just as deliberately, no lone systemctl restart ssh.socket, even though that command appears in many guides. On Debian 13 and Debian 12 it aborts with Job failed. See journalctl -xe for details., and the journal shows ssh.socket: Socket service ssh.service already active, refusing. The new configuration does not become active, and the counter test still reports Permission denied (publickey,password). On Ubuntu 22.04 and Debian 11 it does run through without an error message, but it stops ssh.service and switches the host to socket activation. Because ssh.socket stays disabled there, ssh.service takes over again on the next reboot, so the operating mode flips back and forth unnoticed. try-restart on both units avoids both problems.

Deliberately no reload either: on a system where ssh.socket holds the port, a systemctl reload ssh answers with

fatal: Cannot bind any address.

After that the service sits in a failed state and has let go of its port. A restart does not have this problem, and it does not terminate existing sessions either.

If you want to move the port, the next distribution difference comes up. Ubuntu 24.04 generates the socket configuration from sshd_config via a systemd generator, so once the port has changed, this is enough:

sudo systemctl daemon-reload
sudo systemctl try-restart ssh.socket ssh.service

If, on the other hand, you deliberately switched Debian over to ssh.socket, the port sits in the unit itself and a change in sshd_config has no effect. You set it through a drop-in file with sudo systemctl edit ssh.socket, using an empty ListenStream= to reset it and a second line with the new value. Afterwards, check the result against reality, not against the configuration:

sudo ss -tlnp

Proof: sshd -T and the counter test

A command that runs through without an error is not proof. The proof is the resolved overall configuration, with every included file already accounted for:

sudo sshd -T | grep -E '^(passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|permitrootlogin|usepam|port|authorizedkeysfile)'

The expected output looks like this:

port 22
permitrootlogin prohibit-password
pubkeyauthentication yes
passwordauthentication no
kbdinteractiveauthentication no
usepam yes
authorizedkeysfile .ssh/authorized_keys .ssh/authorized_keys2

If it says passwordauthentication yes there despite your file, then the directory holds a file that sorts alphabetically before yours. Back to the section about ordering.

For a single query, the same command with a narrower filter is enough:

sudo sshd -T | grep -i passwordauthentication

Two reasons speak for the sudo in front of it. First, sshd -T reads the host keys, which under /etc/ssh are readable by root only. Second, sshd itself lives in /usr/sbin, and on Debian that directory is not part of a normal user's PATH, which is why the call ends there with sshd: command not found when sudo is missing. If you want to avoid the route through sudo, write out the full path /usr/sbin/sshd.

From OpenSSH 9.3 onwards, so on Ubuntu 24.04 (9.6p1) and Debian 13 (10.0p2), there is also sshd -G. The option evaluates the same configuration, but it requires neither readable host keys nor an existing /run/sshd, which makes it useful for checks in automation and containers. On Debian 12 (9.2p1), Ubuntu 22.04 (8.9p1) and Debian 11 (8.4p1) the switch does not exist yet, and sshd rejects it as an unknown option. For instructions that hold everywhere, sudo sshd -T therefore remains the right choice.

Now the counter test, and it has to come from a new terminal while the old session stays open. Force a password login and switch keys off for this attempt:

ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password,keyboard-interactive root@203.0.113.10

It is correct when you are rejected immediately and without any password prompt at all:

root@203.0.113.10: Permission denied (publickey).

What matters is what stands inside the parentheses. If it says (publickey,password) there, or if a password prompt appears, password authentication is still open. Only once the counter test fails cleanly and a normal key connection still succeeds do you close the old session and stop the timed rollback:

sudo systemctl stop ssh-rollback.timer

Error messages, word for word

Permission denied (publickey). The server accepts keys only, and yours does not match. Run ssh -v and look at which file was offered in the first place. The most common causes: wrong username, key in the authorized_keys of the wrong user, or you have locked root out and are still logging in as root.

Authentication refused: bad ownership or modes for directory /home/hani/.ssh This line is not on your screen but in the server's log, visible through sudo journalctl -u ssh -n 50 --no-pager. Do not filter with -t sshd here: from OpenSSH 9.8 onwards, which on Debian 13 is the shipped state, sessions run in their own sshd-session process, and the sshd tag then carries only listener messages and not a single login. If you want to filter by tag, use sudo journalctl -t sshd -t sshd-session -n 50 --no-pager. OpenSSH refuses keys when the home directory, .ssh or authorized_keys are writable by group or others. The fix:

chmod go-w ~ && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys

WARNING: UNPROTECTED PRIVATE KEY FILE! or Load key "/home/hani/.ssh/id_ed25519": bad permissions. The same problem on the client side. chmod 600 ~/.ssh/id_ed25519 solves it.

Too many authentication failures Your agent offers every loaded key in turn and exceeds MaxAuthTries along the way. Restrict the connection to one single key: ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519 hani@203.0.113.10.

sign_and_send_pubkey: no mutual signature supported An old RSA key with an SHA-1 signature meets a server that no longer accepts it. Generate an ed25519 key instead of tinkering with PubkeyAcceptedAlgorithms.

Bad owner or permissions on C:\Users\hani\.ssh\config The Windows client checks the permissions of its configuration file. In the file properties under Security, remove inheritance and every entry except your own user account and SYSTEM.

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! The server's host key is a different one than last time. After a reinstall that is to be expected, otherwise it is not. Remove the old entry specifically with ssh-keygen -R 203.0.113.10, and only if you know the reason.

Locked out: the route through the console in the customer panel

If both sessions are gone and no connection comes up any more, that is not data loss, just a detour. Every KVM root server and every dedicated server at KernelHost has a console in the customer panel that sits on the system's screen output and is independent of the server's network stack. So you get in even when SSH is no longer listening at all.

1. Log in to the customer panel, open the affected server and start the console.
2. At the login prompt, log in as root with the password assigned during provisioning. The console login does not go through SSH and is not affected by PermitRootLogin.
3. Undo the change: sudo rm /etc/ssh/sshd_config.d/01-hardening.conf
4. Check and restart: sudo sshd -t && sudo systemctl try-restart ssh.socket ssh.service
5. If SSH is not running at all, sudo systemctl status ssh.socket ssh.service and a look at sudo journalctl -u ssh -n 50 --no-pager will help. A return value of 3 from status only means that one of the two units is inactive, which on Debian, with ssh.socket disabled, is the normal case.

Two precautions spare you this detour almost every time. Write down the root password before you switch password authentication off, because the console needs it. And check for an active firewall before you move the SSH port. A port change without a matching rule locks you out just as reliably as a broken sshd_config, but it looks different: instead of Permission denied you get Connection timed out.

Short version

• Leave a second session open until a third, new connection is proven to work.
• ed25519 with -a 100 and a passphrase, public key via ssh-copy-id, on Windows through a pipe.
• Test key authentication before password authentication goes away.
• Hardening goes into /etc/ssh/sshd_config.d/01-hardening.conf, not into sshd_config. The first value found wins, hence the low number.
• Do not forget KbdInteractiveAuthentication no, otherwise the PAM detour stays open.
• Activate with sudo sshd -t && sudo systemctl try-restart ssh.socket ssh.service, neither with reload nor with a lone restart ssh.socket.
• Clarify beforehand with systemctl is-enabled ssh.socket ssh.service which unit is active in the first place. Ubuntu 24.04 uses the socket, Debian 13 and Debian 12 use the service.
• Proof through sudo sshd -T and a forced password attempt from a new session.
• The emergency route is the console in the customer panel, so keep the root password to hand.

The obvious next layers are described in our articles on fail2ban and on the UFW firewall. More important than either is what you have just finished.