Introduction

SSH (Secure Shell) is essential for securely managing remote Linux servers. This guide walks through installing OpenSSH Server on Ubuntu, configuring SSH key authentication, and applying security best practices.

What you’ll learn:

  • Install and configure OpenSSH Server
  • Set up SSH key-based authentication
  • Disable password authentication for security
  • Configure SSH client for convenience
  • Troubleshoot common SSH issues

Prerequisites

  • Ubuntu Server installed (18.04 LTS, 20.04 LTS, 22.04 LTS, or 24.04 LTS)
  • sudo/root access on the Ubuntu server
  • Local computer with SSH client (Linux/macOS have it built-in, Windows uses PowerShell or PuTTY)
  • Network connectivity between your computer and the server

Part 1: Install OpenSSH Server

SSH into the VM console or use the Proxmox console to run these commands.

Step 1: Update Package Lists

sudo apt update

Step 2: Install OpenSSH Server

sudo apt install -y openssh-server

Expected output:

Reading package lists... Done
Building dependency tree... Done
openssh-server is already the newest version (1:8.9p1-3ubuntu0.4).
The following packages were automatically installed and are no longer required:
  ...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Step 3: Verify SSH is Running

sudo systemctl status ssh

Expected output:

● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2024-12-06 10:00:00 UTC; 2min ago

SSH should automatically start and be enabled to start on boot.

Step 4: Enable SSH on Boot (if not already)

sudo systemctl enable ssh

Part 2: Find Your Server’s IP Address

Before connecting via SSH, you need the server’s IP address.

# Show IP address
ip a

Example output:

2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether bc:24:11:2e:7f:90 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic ens18

In this example, the IP address is 192.168.1.110.

Alternative command:

hostname -I

Part 3: Test SSH Connection

From your local computer, test the SSH connection:

ssh [email protected]

Replace:

  • username with your Ubuntu username (e.g., admin, patrickpriestley)
  • 192.168.1.110 with your server’s actual IP

First-time connection:

The authenticity of host '192.168.1.110 (192.168.1.110)' can't be established.
ED25519 key fingerprint is SHA256:abcd1234efgh5678ijkl9012mnop3456qrst7890uvwx.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Warning: Permanently added '192.168.1.110' (ED25519) to the list of known hosts.
[email protected]'s password:

Type yes and press Enter, then enter your password.

Successful login:

Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-91-generic x86_64)
...
Last login: Mon Dec  6 10:15:00 2024 from 192.168.1.50
username@ubuntu-server:~$

Part 4: Set Up SSH Key Authentication

SSH keys are more secure than passwords and allow passwordless login.

Step 1: Create SSH Directory on Server

# On the Ubuntu server
mkdir -p ~/.ssh
chmod 700 ~/.ssh

Step 2: Generate SSH Key (on your local computer)

Linux/macOS:

# On your local computer
ssh-keygen -t ed25519 -C "[email protected]"

Windows (PowerShell):

ssh-keygen -t ed25519 -C "[email protected]"

Prompts:

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519):

Press Enter to accept default location.

Enter passphrase (empty for no passphrase):
Enter same passphrase again:

Recommendation: Use a strong passphrase for added security.

Expected output:

Your identification has been saved in /home/user/.ssh/id_ed25519
Your public key has been saved in /home/user/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:abcd1234efgh5678ijkl9012mnop3456qrst7890uvwx [email protected]

Step 3: Copy Public Key to Server

Method A: Using ssh-copy-id (Easiest)

# From your local computer
ssh-copy-id [email protected]

Enter your password when prompted. The public key will be automatically added to ~/.ssh/authorized_keys on the server.

Expected output:

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s)
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

Method B: Manual Copy (if ssh-copy-id unavailable)

# On your local computer, display public key
cat ~/.ssh/id_ed25519.pub

Copy the output (starts with ssh-ed25519 AAAA...).

Then on the server:

# On the server
echo "ssh-ed25519 AAAA...your-public-key..." >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Step 4: Test Key-Based Login

# From your local computer
ssh [email protected]

You should now log in without entering your password (just the SSH key passphrase if you set one).

Part 5: Secure SSH Configuration

Step 1: Disable Password Authentication

Once SSH keys work, disable password login for security:

# On the server
sudo nano /etc/ssh/sshd_config

Find and modify these lines:

PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no

⚠️ IMPORTANT: Test SSH key login works BEFORE disabling password authentication!

Step 2: Disable Root Login

PermitRootLogin no

Port 2222

Note: Remember to use -p 2222 when connecting: ssh -p 2222 username@ip

Step 4: Restart SSH Service

sudo systemctl restart ssh

Step 5: Test Configuration

Before logging out, test the new configuration in a new terminal:

ssh [email protected]

If it works, close your old session. If not, fix the config before closing your current session.

Part 6: SSH Client Configuration (Convenience)

Create an SSH config file on your local computer to simplify connections.

Create/Edit SSH Config

# On your local computer
nano ~/.ssh/config

Add an entry for your server:

Host ubuntu-server
    HostName 192.168.1.110
    User admin
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3

Now you can connect with:

ssh ubuntu-server

Much easier than remembering IP addresses!

Troubleshooting

Permission Denied (publickey)

Cause: SSH key not properly installed or permissions wrong

Solution:

# On server, check permissions
ls -la ~/.ssh
# Should show:
# drwx------ ~/.ssh (700)
# -rw------- ~/.ssh/authorized_keys (600)

# Fix permissions if needed
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Connection Refused

Cause: SSH service not running or firewall blocking

Solution:

# Check if SSH is running
sudo systemctl status ssh

# Start if stopped
sudo systemctl start ssh

# Check firewall
sudo ufw status
sudo ufw allow 22/tcp  # or 2222 if you changed the port

Connection Times Out

Cause: Wrong IP address, network issue, or firewall

Solution:

# Verify IP address on server
ip a

# Ping the server from local computer
ping 192.168.1.110

# Check if port is open (from local computer)
telnet 192.168.1.110 22

Too Many Authentication Failures

Cause: SSH client trying too many keys

Solution:

# Specify exact key to use
ssh -i ~/.ssh/id_ed25519 [email protected]

# Or add to ~/.ssh/config:
IdentitiesOnly yes

Security Best Practices

  1. Always use SSH keys instead of passwords
  2. Disable password authentication after setting up keys
  3. Disable root login via SSH
  4. Use strong passphrases on SSH keys
  5. Change default port to reduce automated attacks
  6. Use fail2ban to block brute-force attempts:
    sudo apt install fail2ban
sudo systemctl enable fail2ban
  7. Keep SSH updated:
    sudo apt update && sudo apt upgrade openssh-server

Next Steps

  • Set up static IP address for consistent server access
  • Configure UFW firewall
  • Install Docker or other services
  • Set up SSH agent forwarding for multi-hop connections
  • How to Create a VM in Proxmox
  • How to Configure Static IP on Ubuntu with Netplan
  • How to Install Docker on Ubuntu Server

Last updated: November 2025