You just set up a new Linux server. The first thing you do is log in with a password. Then you hear the horror stories — brute-force attacks, credential stuffing, a whole industry dedicated to guessing your root password. Password-based SSH is honestly a disaster waiting to happen.

The fix is straightforward: create an RSA key on Linux and switch to key-based authentication. You generate a key pair, put the public half on your server, keep the private half on your machine, and suddenly your server stops caring about passwords entirely. No password, no brute force. Simple.

This guide walks you through the entire process — from generating the key pair to deploying it on a remote server, plus some tips that most tutorials skip. If you're running a VPS (which defaults to SSH key login out of the box, smart choice), this guide is especially relevant since you'll need to understand keys from day one.

• A Linux machine (local computer, WSL on Windows, or macOS — the commands are the same)
• A remote Linux server to connect to
• openssh-client installed (it almost certainly already is)

To check:

bash ssh -V

You should see something like OpenSSH_9.x. If not, install it:

bash

sudo apt install openssh-client

sudo dnf install openssh-clients

Open your terminal on the local machine (not the server). Run:

bash ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Breaking this down:

• -t rsa — specify RSA algorithm
• -b 4096 — use 4096-bit key length (stronger than the 3072-bit default)
• -C "your_email@example.com" — a label/comment to identify the key, usually your email

The terminal will ask you three things:

1. Where to save the key:

Enter file in which to save the key (/home/you/.ssh/id_rsa):

Press Enter to accept the default (~/.ssh/id_rsa). If you're managing multiple servers with different keys, type a custom name like ~/.ssh/id_rsa_dmit_vps.

2. Passphrase:

Enter passphrase (empty for no passphrase):

This encrypts your private key file. If someone steals the file, they still can't use it without the passphrase. Highly recommended — use something long and memorable. Press Enter for no passphrase (less secure but convenient for automated scripts).

3. Confirm passphrase — type it again.

After this, you'll see output like:

Your identification has been saved in /home/you/.ssh/id_rsa Your public key has been saved in /home/you/.ssh/id_rsa.pub The key fingerprint is: SHA256:K7tB2qY9X3mZ1vN8pL4sR6wE2qT5yU8iO9pA3sD7fG0 you@example.com

Done. You now have two files:

Before copying the key, take a quick look at it:

bash cat ~/.ssh/id_rsa.pub

It'll look something like:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKhL7... you@example.com

This is what you'll copy to the server. The private key (id_rsa) is much longer and starts with -----BEGIN OPENSSH PRIVATE KEY-----.

There are two ways to do this.

bash ssh-copy-id -i ~/.ssh/id_rsa.pub username@your_server_ip

This command automatically appends your public key to ~/.ssh/authorized_keys on the remote server. You'll need to authenticate with your password one last time for this to work.

Example:

bash ssh-copy-id -i ~/.ssh/id_rsa.pub root@203.0.113.45

Log in to the server with your password:

bash ssh username@your_server_ip

Then on the server, run:

bash mkdir -p ~/.ssh chmod 700 ~/.ssh echo "paste_your_public_key_here" >> ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys

Replace the echo content with the actual output of cat ~/.ssh/id_rsa.pub from your local machine.

Now test before disabling password auth (you don't want to lock yourself out):

bash ssh -i ~/.ssh/id_rsa username@your_server_ip

If it works without prompting for a server password (your passphrase is fine if you set one), you're in. Key-based authentication is working.

Once you confirm key login works, lock out password-based SSH entirely. On the server, edit the SSH config:

bash sudo nano /etc/ssh/sshd_config

Find and change these lines:

PasswordAuthentication no ChallengeResponseAuthentication no UsePAM no

Save the file, then restart SSH:

bash sudo systemctl restart sshd

From this point, your server won't accept password logins. Only your key gets in.

⚠️ Before restarting sshd: Keep your current SSH session open. Open a second terminal and test key login works. Only then restart sshd. If something goes wrong and you close your only session, you may need console access to recover.

Managing multiple VPS instances gets messy fast. Instead of typing full commands every time, set up an SSH config file:

bash nano ~/.ssh/config

Add entries like:

Host dmit-lax HostName 203.0.113.45 User root IdentityFile ~/.ssh/id_rsa_dmit_vps Port 22

Host hkg-server HostName 198.51.100.20 User admin IdentityFile ~/.ssh/id_rsa_hkg Port 22

Now instead of ssh -i ~/.ssh/id_rsa_dmit_vps root@203.0.113.45, you just type:

bash ssh dmit-lax

Neat. Especially useful if you're running multiple DMIT nodes across LA, Hong Kong, and Tokyo.

Speaking of VPS — if you're using or considering DMIT, here's something worth knowing: DMIT enables SSH key authentication by default on all their VPS instances. When you order a server, you paste in your public key during setup, and password login is disabled from the start. It's one of those security-first decisions that budget providers usually skip.

DMIT runs KVM-based VPS with AMD EPYC processors across four data center locations — Los Angeles, San Jose, Hong Kong, and Tokyo — with a focus on optimized network routing to mainland China (CN2 GIA, CMIN2, CMI). For developers who need a reliable Linux environment to test SSH configurations, build applications, or set up secure remote access infrastructure, DMIT's hardware and network quality make it a solid choice.

👉 Check out DMIT VPS plans

"Permission denied (publickey)"

This is the most common issue. Check:

1. The public key is in ~/.ssh/authorized_keys on the server

2. File permissions are correct: bash chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys

3. You're using the right private key: bash ssh -i ~/.ssh/id_rsa_correct_key username@server_ip

4. The SSH agent has your key loaded: bash ssh-add ~/.ssh/id_rsa ssh-add -l # list loaded keys

"WARNING: UNPROTECTED PRIVATE KEY FILE!"

Your private key file permissions are too open:

bash chmod 600 ~/.ssh/id_rsa

"Could not open a connection to your authentication agent"

Start the SSH agent:

bash eval "$(ssh-agent -s)" ssh-add ~/.ssh/id_rsa

Key exists but login still uses password

The server's sshd_config might still have PasswordAuthentication yes. Check with:

bash sudo grep PasswordAuthentication /etc/ssh/sshd_config

While this guide is about creating RSA keys on Linux (the most widely-searched method), it's worth knowing that Ed25519 is now considered the better choice for new deployments:

To generate an Ed25519 key instead:

bash ssh-keygen -t ed25519 -C "your_email@example.com"

For connecting to legacy systems or anywhere that must support older SSH implementations, stick with RSA 4096. For everything modern — including DMIT VPS instances running current Linux distributions — Ed25519 is slightly faster and equally well-supported.

If you're setting up a Linux VPS to practice SSH key management or run production workloads, here's what DMIT's current lineup looks like. All plans come with KVM virtualization, AMD EPYC processors, enterprise SSD storage, 1 IPv4 + 1 IPv6/64, and SSH key login by default.

Promo code: LAX-EB-LAUNCH-NON-MONTHLY-RECURRING-20OFF — 20% off recurring for quarterly/annual billing

Promo code: SJC-Unmetered-Annually-30OFF — 30% off annual unmetered plans

Promo code: HKG-T1-ANNUALLY-45OFF-RECUR — 45% lifetime discount + upgraded specs (more vCPU, 2x disk, 50%+ more RAM, higher IO) on annual plans

Promo code: 2025-TYO-T1-HI-GSL-NON-MONTHLY-30OFF — 30% lifetime discount on quarterly/annual Tokyo Tier 1 plans

Currently Active Promo Codes (2026):

Note: Promo codes are subject to availability and may change. Verify current validity before ordering.

Creating an RSA key on Linux comes down to four things:

1. Run ssh-keygen -t rsa -b 4096 on your local machine
2. Copy ~/.ssh/id_rsa.pub to the server's ~/.ssh/authorized_keys
3. Test the login before locking anything down
4. Optionally disable password auth in sshd_config

That's really it. The extra steps — SSH config files, multiple key management, passphrase best practices — make life easier as your server count grows, but the core process is simple.

If you're shopping for a Linux VPS that takes security seriously from day one (SSH key auth enabled by default, no overselling, solid CN2 GIA or CMIN2 routing), 👉 DMIT is worth a look. Their entry-level plans start at $36.9/year, and with the active promo codes listed above, the already-reasonable pricing gets noticeably better.