Secure a new Ubuntu server before you put apps on it

A fresh Ubuntu server from any cloud provider comes with one account (root), one way in (a password or a key the provider generated for you), and no firewall rules at all. That’s fine for the first five minutes while you’re getting your bearings, but it’s not a server you should run anything on yet. Anyone scanning the internet for open SSH ports will find it within hours, and a root account reachable by password is exactly what they’re hoping to find. By the end of this guide you’ll have a non-root user with its own SSH key, root login and password authentication both turned off, a firewall that only lets through the ports you actually use, fail2ban watching for brute-force attempts, security updates applying themselves, and a short checklist to confirm all of it actually took effect.

What you’ll need

• A fresh Ubuntu 24.04 LTS server from any provider, with its initial root SSH credentials (a password or a provider-generated key).
• Your own SSH key pair on your machine (ssh-keygen -t ed25519 if you don’t already have one).
• A non-root username you want to create on the server (this guide uses deploy).
• About twenty minutes, most of it spent reading output rather than typing.

Step 1: Create a non-root sudo user

Running everything as root means a single mistyped command, a compromised script, or a leaked key hands over the entire machine. A separate account with sudo gives you the same ability to administer the server, but only when you deliberately ask for it, and it’s the account Railyard (or any deploy tool) will actually use later.

As root:

adduser deploy
usermod -aG sudo deploy

adduser walks you through setting a password and some optional details; usermod -aG sudo adds the new account to the sudo group, which on Ubuntu is already wired up to grant full administrative rights.

Check it worked: run id deploy and confirm sudo appears in the group list. Then su - deploy followed by sudo whoami should prompt for the deploy user’s password and print root.

Step 2: Copy your SSH key over and test login

Before you touch anything in sshd_config, get key-based login working for the new user and confirm it from a brand new terminal window, while your original root session stays open as a safety net.

The easy way, from your own machine:

ssh-copy-id deploy@YOUR_SERVER_IP

If ssh-copy-id isn’t available, do it by hand as the deploy user on the server:

mkdir -p ~/.ssh
echo "ssh-ed25519 AAAA... your-public-key-here" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Now, in a second terminal window, without closing your existing root session, try logging in as the new user:

ssh deploy@YOUR_SERVER_IP

You should see it drop you straight into a shell as deploy (it may ask for your key’s own passphrase if you set one, but not a password for the account). If this fails for any reason, fix it now, still from your original root session. Don’t move on to the next step until this works.

Step 3: Disable root login and password authentication

With key-based login confirmed, root access and password authentication are no longer doors you need open, and leaving them open is what most automated SSH scanners are actually probing for.

Edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes

Before reloading anything, check the file is syntactically valid:

sudo sshd -t

If that comes back clean, reload the service (on Ubuntu the systemd unit is named ssh, not sshd):

sudo systemctl reload ssh

Check it worked from a third, fresh terminal (again, keep your working session open): ssh root@YOUR_SERVER_IP should be refused outright, and ssh deploy@YOUR_SERVER_IP with the wrong password should fail immediately with no password prompt at all, since password authentication is off entirely. Your original key-based login as deploy should still work exactly as before.

Step 4: Set up ufw

A firewall that denies everything by default, then explicitly allows only the ports you’re using, means a port you forgot to close (or a service you installed and forgot about) isn’t reachable from the internet just because it happens to be listening.

sudo apt update
sudo apt install ufw
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

22/tcp keeps SSH reachable (you can use sudo ufw allow OpenSSH instead if you prefer the named application profile), 80/tcp and 443/tcp cover plain HTTP and HTTPS for whatever you’ll eventually deploy. If you know you won’t be serving anything over HTTP, you can drop the 80/tcp rule, though most TLS setups still use port 80 briefly for certificate issuance.

Check it worked: sudo ufw status verbose should show Status: active, Default: deny (incoming), allow (outgoing), and the three rules you just added listed as ALLOW.

Step 5: Install and configure fail2ban

Even with password authentication off, your SSH port still accepts connection attempts, and a steady stream of automated login tries clutters your logs and burns a bit of CPU for no reason. fail2ban watches the auth log and temporarily bans IPs that fail to authenticate too many times.

sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

fail2ban’s own convention is to leave jail.conf untouched (it gets overwritten on package upgrades) and put your changes in jail.local, which is read afterward and takes precedence. Open /etc/fail2ban/jail.local and make sure the sshd section looks like this:

[sshd]
enabled = true
port = ssh
maxretry = 5
bantime = 1h
findtime = 10m

That bans an IP for an hour after five failed attempts within a ten-minute window. Then enable and start the service:

sudo systemctl enable --now fail2ban

Check it worked: sudo fail2ban-client status sshd should show the jail running, with counters for currently failed and currently banned addresses.

Step 6: Turn on unattended security upgrades

Most server compromises aren’t zero-days, they’re known vulnerabilities in packages nobody got around to updating. unattended-upgrades applies security patches automatically, on a schedule, without you having to remember to SSH in and run apt upgrade.

sudo apt install unattended-upgrades

Ubuntu ships this with security updates already enabled for the current release once it’s installed. Confirm (and adjust if needed) the two config files:

/etc/apt/apt.conf.d/20auto-upgrades should contain:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

/etc/apt/apt.conf.d/50unattended-upgrades lists which package origins are allowed to auto-update; the security pocket for your release is included by default.

Check it worked: sudo unattended-upgrade --dry-run --debug prints what it would do on its next run without actually changing anything. After the service has run for real (it checks once a day), /var/log/unattended-upgrades/unattended-upgrades.log shows what it installed.

Step 7: Set the server’s timezone and hostname

This one isn’t about attackers, it’s about your own sanity later: log timestamps that match your own clock, and a hostname that tells you which server you’re looking at when you have more than one open in different terminal tabs.

sudo timedatectl set-timezone America/Montreal
sudo hostnamectl set-hostname myapp-server-01

Swap in whichever IANA timezone and hostname make sense for you; timedatectl list-timezones lists the valid values if you’re not sure of the exact name.

Check it worked: timedatectl shows the timezone you set under “Time zone”, and hostnamectl shows the new hostname under “Static hostname”.

Step 8: Final verification pass

Before you consider the server ready, log all the way out and confirm the whole setup holds up from a clean slate, not just from the session you’ve had open the whole time:

• Open a brand new terminal and run ssh deploy@YOUR_SERVER_IP. You should land in a shell using only your key, no password prompt.
• ssh root@YOUR_SERVER_IP from that same fresh terminal should be refused.
• sudo ufw status verbose still shows active with exactly the ports you intend open.
• sudo fail2ban-client status sshd shows the jail running.
• sudo whoami as the deploy user still returns root after a password prompt.

If every one of those checks out, the server is in a reasonable state to put something on.

The shortcut: do it with Railyard

None of the steps above are things Railyard does for you. It doesn’t touch sshd_config, it doesn’t configure ufw or fail2ban, and it won’t set your timezone. What it needs from a server you’ve already hardened the way above is narrow: the SSH user has passwordless sudo, ports 22, 80 and 443 are reachable from the internet (exactly the rules you just added), and the server has at least 1 GB of RAM (2 GB or more if you’re building app images on it too).

One detail worth fixing before you connect a server: the default adduser plus usermod -aG sudo combination from Step 1 still asks deploy for its password every time it runs sudo, and Railyard needs that to happen without a prompt. Add a sudoers drop-in for it:

echo "deploy ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/deploy
sudo chmod 440 /etc/sudoers.d/deploy

From there: Servers → Add a server → Bring your own server, point it at this server’s IP with the private key matching the public key you copied over in Step 2. That private key is used once for the install and isn’t stored. The server shows up in your dashboard within a few minutes, ready to deploy an app to.

If you’d rather not provision a box by hand at all, Railyard can also create a server directly on your own cloud account (DigitalOcean, Vultr, Hetzner, AWS or OVH) and set it up automatically, appearing in your dashboard the same way. Either path still leaves the hardening above as your responsibility to keep in place: Railyard automates app deployment, zero-downtime releases, and database and backup management on top of a server that’s already secured, it’s a shortcut past the deploy tooling, not past the hardening itself.

Troubleshooting

You locked yourself out after disabling password authentication. This happens when PasswordAuthentication no gets reloaded before key-based login was actually confirmed working. If your provider offers a web console or recovery mode, use it to log in locally and either re-enable PasswordAuthentication yes temporarily or fix the key in ~/.ssh/authorized_keys, then retest from a fresh terminal before disabling it again.

ufw is blocking a port your app needs. If something you deploy later needs a port beyond 22, 80 and 443 (a database you intentionally expose, a different app port), add it explicitly: sudo ufw allow 5432/tcp, then re-check with sudo ufw status verbose. It’s usually safer to keep databases off the public internet entirely and reach them over a private network instead, rather than opening another port.

fail2ban banned your own IP. If you mistype a password a few times while testing (or test from a script that retries fast), fail2ban can ban the IP you’re working from. Unban it from a session that’s still connected, or from the server’s console: sudo fail2ban-client set sshd unbanip YOUR_IP.

You can SSH in from the server’s own console but not from the internet. Most cloud providers layer their own network-level firewall (a “security group” or “cloud firewall”) in front of the server, separate from ufw. If a port is open in ufw but still unreachable, check the provider’s firewall settings too, both need to allow the same ports for traffic to get through.

FAQ

Do I need to do this if my provider already has its own firewall? Yes. A provider’s network firewall is a useful first layer, but it doesn’t give you key-only SSH, it doesn’t ban brute-force attempts, and it won’t patch a vulnerable package for you. Treat ufw as a second layer, not a replacement.

What port should I move SSH to? You don’t have to move it. Once root login and password authentication are off and fail2ban is watching, port 22 isn’t a meaningful weak point, a scanner finding an open port 22 can’t do much without a valid key. Moving SSH to a non-standard port mostly just reduces log noise from automated scans; if you do it, remember to update both sshd_config’s Port directive and the matching ufw rule, and test the new port before closing 22.

How often should I update the server? unattended-upgrades applies security patches daily on its own, which covers the updates that matter most. It’s still worth running sudo apt update && sudo apt upgrade by hand every so often for non-security package updates, and rebooting after a kernel update so the new one actually loads.

Do I still need fail2ban if password authentication is already off? It’s still worth running. It cuts down on log noise and the (small) resource cost of constant failed connection attempts, and it applies the same protection to any other service you later add a jail for, not just SSH.

Does this apply to newer Ubuntu LTS releases too? The general approach (a non-root sudo user, key-only SSH, ufw, fail2ban, unattended upgrades) carries over to other recent Ubuntu LTS releases; the commands and file paths in this guide are what’s current for 24.04. If you’re on a different release, it’s worth a quick check against Ubuntu’s own current documentation for anything that’s changed before you fix a hardening step in place.

None of these steps take long on their own, and once they’re done you don’t have to think about them again until the next server. What you get out of it is a machine where a stray scanner finds a closed door instead of an open one, and that’s the baseline every server you run anything real on should start from.