Make SSH use gpg-agent

2020-02-08 - A few days ago I updated this page with information about how to set this up on Catalina. While I was thinking about it I happened across this article which accomplishes the same overall goal, but instead of disabling the macOS com.openssh.ssh-agent LaunchAgent, it creates a symlink with whatever name $SSH_AUTH_SOCK contains, pointing to $HOME/.gnupg/S.gpg-agent.ssh. This is a LOT easier to set up, so this is what I'm doing with my own machines as I upgrade them to Catalina.

Quick setup - CentOS 7

If you're standing at the console of a CentOS 7 machine and need to use your YubiKey to authenticate outbound SSH connections...

sudo yum install gnupg2-smime pcsc-lite
sudo systemctl start pcscd
eval $( gpg-agent --daemon --enable-ssh-support )
export SSH_AUTH_SOCK="$( gpgconf --list-dirs agent-ssh-socket )"

Now you should be good to go.

Pre-requisites

The two obvious dependencies are an SSH client, and gnupg. One or both of these are usually installed on most Linux and macOS machines.

Linux

Most Linux distros come with openssh already installed, however some distros may split the client and server bits into separate packages. Some distros may install gnupg as well - if not, you should be able to use yum, apt-get, or a similar command, to install the necessary packages. Search

CentOS, Fedora, RedHat, etc.

yum install openssh-clients gnupg2 gnupg2-smime

Debian, Ubuntu, etc.

apt-get install (names?)

Others

I don't have the exact commands for every other distro out there. For gnupg you should search for packages with names like gnupg, gpg2, or maybe just gpg.

macOS

For macOS, the openssh client is installed as a basic part of the OS, however gnupg is not. You should visit https://gpgtools.org/ and install the current version of GPG Suite. Not only will this give you the gpg command line tools, but it also includes Mail.app support for signing and encrypting email, and a System Preferences widget to control some aspects of how gpg and gpg-agent work.

Setup - Linux

To make the current shell use gpg-agent (and therefore the YubiKey) instead of the normal ssh-agent ...

Manual process

Make sure the GPG_TTY variable is set.
```
export GPG_TTY=$(tty)
```

Make sure that the SSH_AUTH_SOCK variable points to the S.gpg-agent.ssh socket.

unset SSH_AGENT_PID
export SSH_AUTH_SOCK="$( gpgconf --list-dirs agent-ssh-socket )"

Any commands executed in this shell will use gpg-agent as the SSH agent.

Automatic process (shell, per-user)

To make sure that your shell always sets the GPG_TTY and SSH_AUTH_SOCK variables correctly, add the following to your .bash_profile (or the appropriate file, if your login shell is not bash)

########################################
# Set things up for using gpg-agent

export GPG_TTY=$(tty)

function use-gpg-agent-for-ssh {
    SOCK="$( gpgconf --list-dirs agent-ssh-socket )"
    if [[ -n "${SOCK:-}" ]]
    then
        unset SSH_AGENT_PID
        export SSH_AUTH_SOCK="$SOCK"
    fi
}

use-gpg-agent-for-ssh

This creates a function to "do the work", and then calls that function. This way if you decide you don't want this all the time, you can comment out just the function call (the last line), and then type use-gpg-agent-for-ssh in a shell to easily "activate" the change within that shell.

Once you have added this, every new interactive shell will use the changes. A quick way to test it is to open a new terminal window, which will contain a new shell. Once you have verified that it's working, you can either close the shell you're working in and open a new window, or you can run "source ~/.bash_profile" to read the updated profile into the current shell.

Note that setting the variables in this way will only affect shells and any processes started from those shells. In particular, it will NOT affect processes started by something other than your shell, such as cron jobs.

Automatic process (all users)

The process is the same as the "shell, per-user" process above, except that instead of editing your ~/.bash_profile file...

You will edit /etc/profile, so that all users see it.
On some systems you may be able to create an /etc/profile.d/use-gpg-agent-for-ssh.sh file instead.

If your system has multiple users, and some of them may want to use the normal ssh-agent, you may want to not include calling the function (i.e. the final use-gpg-agent-for-ssh line) in what you add to the system-wide profile. In this case, users who do want to use gpg-agent by default can add a user-gpg-agent-for-ssh line to their ~/.bash_profile, and anybody on the system can manually type that command to use gpg-agent within that shell.

Setup - macOS

NOTE: macOS 10.15 "Catalina" changes a lot of things under the covers. Most of the changes are good things from a security standpoint, however they go out of their way to make it impossible to prevent the standard ssh-agent LaunchAgent from starting, or more specifically, from overriding the SSH_AUTH_SOCK variable before the desktop environment starts. This means that under Catalina, you cannot use SSH keys stored on a Yubikey with other programs (such as BBEdit), unless those apps have ways to override environment variables (like a .bashrc file, but that only works for shells.)

Notes

$ launchctl stop com.openssh.ssh-agent
$ launchctl disable gui/$(id -u)/com.openssh.ssh-agent
$ launchctl unload /System/Library/LaunchAgents/com.openssh.ssh-agent.plist
/System/Library/LaunchAgents/com.openssh.ssh-agent.plist: Operation not permitted while System Integrity Protection is engaged

Symlink

macOS comes with a LaunchAgent which does two things:

Creates a UNIX socket with a dynamic name, and sets things up so that ssh-agent is listening on on that socket.
Exports an SSH_AUTH_SOCK environment variable whose value is the path to that dynamically generated socket.

We are going to create our own LaunchAgent which replaces that socket with a symbolic link pointing to the socket where gpg-agent's SSH agent listens. By doing this, any client which uses the $SSH_AUTH_SOCK value to connect to an SSH agent, will end up talking to gpg-agent.

NOTE: I'll finish writing this at some point over the next few days. For now I'll just include the relevant part of the list of commands I run on a machine after upgrading it to Catalina.

cd ~/Library/LaunchAgents
curl -O https://jms1.net/yubikey/net.jms1.gpg-agent.plist
curl -O https://jms1.net/yubikey/net.jms1.gpg-agent-symlink.plist

reboot

Download links:

net.jms1.gpg-agent.plist - starts gpg-agent
net.jms1.gpg-agent-symlink.plist - replaces the UNIX socket with a symlink

OLD - Disable `ssh-agent`

I no longer use the process described in this section, however I'm leaving it here in case anybody is interested, or if something changes in the future and I end up needing to go back to this process.

The first thing we need to do is disable the ssh-agent that Apple installs with macOS. However, before we can do this, we need to disable SIP (System Integrity Protection).

Disable SIP

 → Restart…
When the boot chime plays, press and hold ⌘-R to enter recovery mode
Utilities → Terminal
```
csrutil disable
exit
```
Press ⌘-Q to exit the Terminal and return to the menu.
 → Restart

Remove the `ssh-agent` LaunchAgent

Open a command line window (Terminal, iTerm2, etc.)
Run csrutil status and make sure it says "disabled".

Disable, unload, and remove the LaunchAgent.

launchctl disable gui/$(id -u)/com.openssh.ssh-agent
launchctl unload /System/Library/LaunchAgents/com.openssh.ssh-agent.plist
sudo mv /System/Library/LaunchAgents/com.openssh.ssh-agent.plist /var/root/

If you're running Catalina and the last command fails, re-mount the root filesystem in read-write mode (this can only be done while SIP is disabled) and try it again.

sudo mount -uw /
sudo mv /System/Library/LaunchAgents/com.openssh.ssh-agent.plist /var/root/

Re-enable SIP

 → Restart…
When the boot chime plays, press and hold ⌘-R to enter recovery mode
Utilities → Terminal
```
csrutil enable
exit
```
Press ⌘-Q to exit the Terminal and return to the menu.
 → Restart

Make sure it worked

After the machine restarts normally, make sure it worked.

$ launchctl list com.openssh.ssh-agent

This should say that it can't find the servce.

$ env | grep SSH

You should not see an SSH_AUTH_SOCK variable (and probably won't see any output at all, just another command prompt.)

Set up `gpg-agent`

Quick version:

cd ~/Library/LaunchAgents
curl -O https://jms1.net/yubikey/net.jms1.gpg-agent.plist
curl -O https://jms1.net/yubikey/net.jms1.gpg-agent-env.plist
launchctl load net.jms1.gpg-agent.plist
launchctl load net.jms1.gpg-agent-env.plist

Make `gpg-agent` start automatically

Create $HOME/Library/LaunchAgents/net.jms1.gpg-agent.plist with the following contents: (adjust the path to gpg-connect-agent as needed)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>net.jms1.gpg-agent</string>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <false/>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/local/MacGPG2/bin/gpg-connect-agent</string>
      <string>/bye</string>
    </array>
  </dict>
</plist>

Tell launchd to use it.

launchctl load net.jms1.gpg-agent.plist

Set the `SSH_AUTH_SOCK` variable automatically

Create $HOME/Library/LaunchAgents/net.jms1.gpg-agent-env.plist with the following contents: (adjust the path to the socket file as needed)

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>net.jms1.gpg-agent-env</string>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <false/>
    <key>ProgramArguments</key>
    <array>
      <string>/bin/launchctl</string>
      <string>setenv</string>
      <string>SSH_AUTH_SOCK</string>
      <string>/Users/jms1/.gnupg/S.gpg-agent.ssh</string>
    </array>
  </dict>
</plist>

Tell launchd to use it.

launchctl load net.jms1.gpg-agent-env.plist

Restart

You will need to either reboot, or log out and log back in, in order to activate these changes.

Make sure it worked

After the machine restarts normally, make sure it worked.

Make sure the variable exists, with the correct value.
```
$ env | grep SSH
SSH_AUTH_SOCK=/Users/jms1/.gnupg/S.gpg-agent.ssh
```
You should see an SSH_AUTH_SOCK variable, pointing to the path listed in the net.jms1.gpg-agent-env.plist file.
Make sure the agent is reachable.
```
$ gpg-connect-agent -v /bye
gpg-connect-agent: closing connection to agent
```
You should just see the message shown above.

Configure `gpg-agent`

macOS 10.13 uses OpenSSH_7.5p1, LibreSSL 2.5.4. Either SSH support is enabled by default, or Apple has configured it to always run by default. Your OS may have done the same, you should probably start by checking whether SSH support is already enabled. To do so:

ps auxww | grep gpg-agent, find the PID
lsof -nP -p PID
Make sure there is a socket open with the name ~/.gnupg/S.gpg-agent.ssh

Enable SSH support

If you need to enable SSH support, or want to go through the exercise to ensure that SSH support is explicitly enabled:

Add this line to ~/.gnupg/gpg-agent.conf
```
enable-ssh-support
```
If you changed the file, you should restart the running gpg-agent process:
```
gpg-connect-agent killagent /bye
gpg-connect-agent /bye
```

The `sshcontrol` file

Ref: http://karl.kornel.us/2016/04/sshcontrol/

THIS STEP IS NOT NECESSARY, at least not for gpg-agent (GnuPG/MacGPG2) 2.2.0.

Find the "keygrip" for the authentication key. (A "keygrip" is another form of fingerprint on the public key plus options, see http://lists.gnupg.org/pipermail/gnupg-users/2012-July/045115.html for details.)

$ gpg --list-keys --with-keygrip 6353320118E1DEA2F38EAE806B2EDC90B5C6DC30
...
pub   rsa4096/0x6B2EDC90B5C6DC30 2017-05-27 [SC]
      6353320118E1DEA2F38EAE806B2EDC90B5C6DC30
      Keygrip = 05D34BB61EA379C3B05817A753FD21E2DA6231E1
uid                   [ultimate] John M. Simpson <jms1@voalte.com>
uid                   [ultimate] John M. Simpson <jms1@jms1.net>
sub   rsa4096/0x297E5961AB566594 2017-05-27 [E]
      Keygrip = 4530A5E8BBB0F6A604E847AB21FBC29D1E44B293
sub   rsa4096/0xBA6C2A169C6C0F60 2017-11-10 [A]
      Keygrip = CD8EEE908C11D30706FC1BB704066180F2F0196E

The value we want is the keygrip of the authentication sub-key, since that was uploaded to the YubiKey.

Add a line to ~/.gnupg/sshcontrol containing that value, with "0" after it (this is a TTL value, not really sure how this works). Comments can be added using "#".

# YubiKey 2017-11-10
CD8EEE908C11D30706FC1BB704066180F2F0196E 0

Usage

If you've gone through the setup process above, and the SSH_AUTH_SOCK variable points to the S.gpg-agent.ssh socket, you don't really need to do anything differently - just use ssh, scp, sftp, or whatever, the same way you already do. As long as your SSH client works with an agent, it should "just work".

If you haven't gone through the steps above ... do so.

`authorized_keys`

Getting the public key line needed for authorized_keys files is very easy:

Make sure gpg-agent is running, and the current shell is configured to use it.
```
$ export GPG_TTY=$(tty)
```

If you have NOT done the setup procedure to ensure that ssh always uses gpg-agent...

$ unset SSH_AGENT_PID
$ export SSH_AUTH_SOCK="$( gpgconf --list-dirs agent-ssh-socket )"

Insert the YubiKey, wait a few seconds, and run ssh-add -L. The card's key will have cardno: with the YubiKey's serial number as the comment.

$ ssh-add -L
ssh-rsa AAAAB3NzaC1yc...9toFRmxejrbw== cardno:000606940447

References

http://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-mac-os-x-launchd-plist/ - shows how to create a .plist which runs launchctl setenv during login (since launchctl setenv doesn't appear to persist across reboots.)
Login Script - this script is executed as root, during every user's login process, before the user is able to interact with their account.
- There can only be one login script, shared by all users.
- This is not directly related to SSH using gpg-agent, I just thought it was interesting and decided to make a note of it.

Notes

The gpg-agent automatically "contains" the Authentication Keys stored on the YubiKeys (or other OpenPGP cards) present on the system. When gpg-agent receives an authentication request, it passes it along to the YubiKey, which does the work of signing the request without sending the secret key anywhere.

Other keys can be added to the agent using ssh-add as expected. When you do this, a copy of the secret key will be written to a file in the ~/.gnupg/private-keys-v1.d/ directory, named after the "key grip" (another kind of fingerprint, which includes the options rather than just the public key). These files are encrypted using a passphrase which may or may not be the same as the key's normal passphrase. When you add a key, you will be prompted first for the existing passphrase (to read the secret key), and then for a new passphrase (to encrypt the secret key in this new file).

However, ssh-add -d (or -D) will not remove them (at least, it won't make them stop appearing in ssh-add -l output.) gpg-agent adds the key grips (similar to a Key ID) to a file called ~/.gnupg/sshcontrol. Removing the keygrip from this file makes the key no loger appear in the "ssh-add -l" output, and no longer be available for SSH authentication. (Removing the ~/.gnupg/sshcontrol file will make ALL keys no longer appear in the "ssh-add -l" output.) Note that editing or removing this file does not remove the private-keys-v1.d file - you will need to remove those by hand.

Make SSH use gpg-agent

Quick setup - CentOS 7

Pre-requisites

Linux

CentOS, Fedora, RedHat, etc.

Debian, Ubuntu, etc.

Others

macOS

Setup - Linux

Manual process

Automatic process (shell, per-user)

Automatic process (all users)

Setup - macOS

Notes

Symlink

OLD - Disable ssh-agent

Disable SIP

Remove the ssh-agent LaunchAgent

Re-enable SIP

Make sure it worked

Set up gpg-agent

Make gpg-agent start automatically

Set the SSH_AUTH_SOCK variable automatically

Restart

Make sure it worked

Configure gpg-agent

Enable SSH support

The sshcontrol file

Usage

authorized_keys

References

Notes

OLD - Disable `ssh-agent`

Remove the `ssh-agent` LaunchAgent

Set up `gpg-agent`

Make `gpg-agent` start automatically

Set the `SSH_AUTH_SOCK` variable automatically

Configure `gpg-agent`

The `sshcontrol` file

`authorized_keys`