Setting up SSH keys to work in Windows

Kelvin Kang

Kelvin Kang

— 4 min read

I've typically connected to my servers via a WSL container in Windows. I wanted a cleaner way of getting to my machines without having to first install WSL and a Linux distribution. Here are the steps I took to set it up.

To connect from Windows to Linux or other SSH servers using keys instead of passwords, configure the built‑in OpenSSH client plus (optionally) the SSH agent. This guide walks through generating/using a key, configuring the Windows client, and troubleshooting common issues.

1. Enable OpenSSH client on Windows

On Windows 10/11, the OpenSSH client is usually installed, but you can verify it:

  • Open Settings → Apps → Optional features.
  • Look for OpenSSH Client under “Installed features”. If it is missing, click Add a feature, search for OpenSSH Client, and install it.

After installation, ssh, ssh-keygen, and ssh-add are available in PowerShell, Command Prompt, and Windows Terminal.

You can confirm with:

ssh -v

You should see an output similar to:

OpenSSH_for_Windows_9.xpY, LibreSSL 3.x.y

2. Create or import your SSH key

All SSH keys live in a .ssh directory in your user profile:

C:\Users\<YourUser>\.ssh\

If the directory does not exist, create it.

Option A: Generate a new key

From PowerShell or Windows Terminal:

ssh-keygen -t ed25519 -C "[email protected]"
  • When prompted for the file, press Enter to accept the default
C:\Users\<YourUser>\.ssh\id_ed25519
  • Enter a passphrase for better security (recommended), or leave it empty if you understand the trade‑offs.

This creates:

  • id_ed25519 – private key (keep this secret)
  • id_ed25519.pub – public key (this goes to the server)

Option B: Import an existing key

If you already have a private key file (for example from a Linux or macOS machine):

  1. Copy the private key into C:\Users\<YourUser>\.ssh\.
  2. Make sure the file looks like an SSH private key, for example:text--—BEGIN OPENSSH PRIVATE KEY-----
    ...
    --—END OPENSSH PRIVATE KEY-----
  3. Give it a simple name, e.g. personal.txt or id_rsa.

If your key is in PuTTY’s .ppk format, you must convert it to OpenSSH format with PuTTYgen (Conversions → Export OpenSSH key).

3. Install your public key on the server

On the Linux/Unix server, log in using whatever method you currently use (password or console) and add your public key to authorized_keys:

  1. On Windows, display the public key:powershelltype $env:USERPROFILE\.ssh\id_ed25519.pub
  2. On the server, create the .ssh directory (if needed) and append the key:bashmkdir -p ~/.ssh
    chmod 700 ~/.ssh
    echo 'ssh-ed25519 AAAA... your-comment' >> ~/.ssh/authorized_keys
    chmod 600 ~/.ssh/authorized_keys
  3. Repeat per account or server as required.

From this point, the server will accept that key for logins to that user account.

4. Configure the SSH client on Windows

The per‑user SSH config file lives at:

text

C:\Users\<YourUser>\.ssh\config

If it does not exist, create it as a plain text file named config with no extension.

Here is a sample configuration for a host:

textHost checkpoint-bastion.coburgcrescent.lan
HostName checkpoint-bastion.coburgcrescent.lan
User kelvin
IdentityFile ~/.ssh/personal.txt
IdentitiesOnly yes

Explanation:

  • Host – shortcut name you will use with the ssh command.
  • HostName – DNS name or IP of the server.
  • User – remote username.
  • IdentityFile – path to the private key for this host. On Windows, ~ expands to C:\Users\<YourUser>. Using ~ avoids issues with spaces in C:\Users\First Last\....
  • IdentitiesOnly yes – instructs SSH to use only the keys explicitly listed with IdentityFile for this host instead of trying every default key.

You can add multiple host blocks:

Host bastion
HostName checkpoint-bastion.coburgcrescent.lan
User kelvin
IdentityFile ~/.ssh/personal.txt
IdentitiesOnly yes

Host db
HostName 10.10.1.20
User postgres
IdentityFile ~/.ssh/id_ed25519
IdentitiesOnly yes

Now you can connect simply with:

ssh bastion

5. Use the SSH agent (optional, for convenience)

The SSH agent stores decrypted keys in memory so you do not have to type your passphrase every time.

Start and enable the agent service

Run PowerShell as Administrator and execute:

Get-Service ssh-agent | Set-Service -StartupType Automatic -PassThru | Start-Service

This starts the agent and ensures it starts at boot.

Add your key to the agent

In a non‑admin PowerShell or Windows Terminal window:

ssh-add $env:USERPROFILE\.ssh\personal.txt

You will be prompted for the key’s passphrase once. After that, the agent holds the key.

Verify that the agent has the key:

ssh-add -l

You should see a line showing the fingerprint and comment (e.g. personal.txt ([email protected])).

With the agent running, plain ssh bastion will use your loaded key without requiring the path or passphrase again (until you log out or reboot, depending on your setup).

6. Troubleshooting common issues

“Permission denied (publickey)”

If you see:

Permission denied (publickey).

check the following:

  1. Config file is being read
    Run:powershellssh -v bastion
    In the output, you should see:textdebug1: Reading configuration data C:\\Users\\<YourUser>/.ssh/config
    debug1: C:\\Users\\<YourUser>/.ssh/config line X: Applying options for bastion
    If you do not see this, the config file is in the wrong place or named incorrectly.
  2. IdentityFile is actually used
    In verbose output you should see something like:textdebug1: identity file C:\\Users\\<YourUser>/.ssh/personal.txt type 0
    debug1: Offering public key: C:\\Users\\<YourUser>/.ssh/personal.txt
    If you only see attempts for id_rsa, id_ed25519, etc., and not your custom file, double‑check:
    • Spelling: IdentityFile (not IdentifyFile).
    • Path: use ~/.ssh/filename to avoid quoting and spaces.
    • The host block matches exactly the host you use on the command line.
  3. Agent has no identities
    If verbose output shows:get_agent_identities: agent contains no identities
    then either:Fix by:ssh-add ~/.ssh/personal.txt
    ssh-add -l
    • The agent is not running, or
    • You have not run ssh-add in this session.
  4. Server authorized_keysIf the client clearly offers the key but the server still denies access:
    • Confirm ~/.ssh/authorized_keys on the server contains the correct public key (not the private key).
    • Check file permissions:bashchmod 700 ~/.ssh
      chmod 600 ~/.ssh/authorized_keys
    • Check server logs (journalctl -u ssh or /var/log/auth.log) to see why the key was rejected.

“Error loading key …: invalid format”

This error when running ssh-add means the file is not recognized as a private key. Typical causes:

  • The file contains a password or random text.
  • You used a .pub file (public key) instead of the private key.
  • The file is PuTTY .ppk format and needs conversion to OpenSSH.

Fix by:

  • Ensuring you pass the private key (e.g. personal.txt or id_ed25519) to ssh-add.
  • Converting .ppk with PuTTYgen (Conversions → Export OpenSSH key).

7. Minimal working example

Given a private key stored as C:\Users\Kelvin Kang\.ssh\personal.txt and a Linux server at checkpoint-bastion.coburgcrescent.lan with your public key in ~kelvin/.ssh/authorized_keys, this minimal setup should work:

C:\Users\Kelvin Kang\.ssh\config

Host checkpoint-bastion.coburgcrescent.lan
HostName checkpoint-bastion.coburgcrescent.lan
User kelvin
IdentityFile ~/.ssh/personal.txt
IdentitiesOnly yes

Connection command

ssh checkpoint-bastion.coburgcrescent.lan

If you prefer using the agent:

ssh-add $env:USERPROFILE\.ssh\personal.lan
ssh checkpoint-bastion.coburgcrescent.lan

With this setup, Windows uses your SSH keys reliably to connect to your servers with minimal friction.

Read more