ssh-agent: eval hangs without --deterministic-shutdown

[glitsj16]

I've had deterministic-shutdown in my ssh-agent.local ever since the option was introduced. Never realised until accidentally bumping into this today that the current ssh-agent.profile is likely broken for most users.

The bad...

$ eval "$(firejail /usr/bin/ssh-agent -s)"
[hangs indefinately]

The good...

$ eval "$(firejail --deterministic-shutdown /usr/bin/ssh-agent -s)"
Agent pid 7
$

[rusty-snake]

Agent pid 7

This sounds like it reports the pid inside firejail and not the pid of the initial pid-namespace.

[rusty-snake]

$ eval "$(firejail --deterministic-shutdown /usr/bin/ssh-agent -s)"

This works because it kills the ssh-agent daemon process I guess.

[glitsj16]

This sounds like it reports the pid inside firejail and not the pid of the initial pid-namespace.

Good observation. I'll do more testing and compare what SSH_AGENT_PID and SSH_AUTH_SOCK contain with/without firejail.

[glitsj16]

UPDATE

Even when the 'hanging issue' can be fixed via --deterministic-shutdown, firejailing ssh-agent always results in the running agent being unreachable. Likely this is due to firejail's use of pid namespaces and AFAIK cannot be avoided. Hence we should NEVER sandbox ssh-agent.

I'm closing #5752 and #5753 as both PR's have lost their relevancy in this context IMO.

FWIW we don't need to do anything for firecfg, as #1568 already keeps it out. BUT we do need to consider dropping ssh-agent.profile, something that rarely happens and should be made clear somehow. Adding a note in RELNOTES might be the best we can do here.

[kmk3]

@glitsj16 on Mar 30:

Even when the 'hanging issue' can be fixed via --deterministic-shutdown,
firejailing ssh-agent always results in the running agent being unreachable.
Likely this is due to firejail's use of pid namespaces and AFAIK cannot be
avoided. Hence we should NEVER sandbox ssh-agent.

Does this affect only the ssh-agent $command case or the general case of
running ssh-agent + ssh-add + some program? Could you post an example?

Wouldn't both sides having access to the socket be enough regardless of any
other namespaces?

If the only problem is the PID namespace, wouldn't it be enough to use --join
for the applications that should access the given ssh-agent instance?

This might even be a good way to have multiple ssh-agent instances running with
different keys being used for different programs.

Examples of theoretically isolated groups (using PID:program-name):

• sandbox1: 100:ssh-agent (with "foo" key) + 101:IDE (personal)
• sandbox2: 200:ssh-agent (with "bar" key) + 201:IDE (work)

FWIW we don't need to do anything for firecfg, as #1568 already keeps it
out.

Good to know.

BUT we do need to consider dropping ssh-agent.profile, something that
rarely happens and should be made clear somehow. Adding a note in RELNOTES
might be the best we can do here.

Yes; that was done for nvm.profile. If ssh-agent.profile is to be dropped, I
think just adding it to the RELNOTES would be fine, especially considering that
it's already been disabled in firecfg.config for a while.

[glitsj16]

Does this affect only the ssh-agent $command case or the general case of
running ssh-agent + ssh-add + some program? Could you post an example?

It's the general case that's broken:

$ eval "$(firejail --deterministic-shutdown /usr/bin/ssh-agent -s)"
Agent pid 11
$ echo $SSH_AUTH_SOCK
/tmp/ssh-XXXXXXaUfxD4/agent.10
$ echo $SSH_AGENT_PID
11
$ ssh-add -l
Error connecting to agent: No such file or directory

This works:

$ firejail --noblacklist=/tmp/.X11-unix /usr/bin/ssh-agent xterm
[glitsj16@lab Downloads]$ ssh-add -l
The agent has no identities.

Although I can understand your reasoning, how could one --join the ssh-agent sandbox if that requires --deterministic-shutdown? That's a use case I can't get working here, but I'm always open to suggestions.

Luckily there are several other SSH agents that do work when firejailed.

[kmk3]

@glitsj16 on Mar 30:

Does this affect only the ssh-agent $command case or the general case of
running ssh-agent + ssh-add + some program? Could you post an example?

It's the general case that's broken:

$ eval "$(firejail --deterministic-shutdown /usr/bin/ssh-agent -s)"
Agent pid 11
$ echo $SSH_AUTH_SOCK
/tmp/ssh-XXXXXXaUfxD4/agent.10
$ echo $SSH_AGENT_PID
11
$ ssh-add -l
Error connecting to agent: No such file or directory

Ah I see what the problem is now.

--deterministic-shutdown makes ssh-agent be gone right after running it.

There is nothing running as $SSH_AGENT_PID and no file at $SSH_AUTH_SOCK,
so ssh-add can't do anything:

$ eval "$(firejail --deterministic-shutdown /usr/bin/ssh-agent -s)"
Agent pid 5
$ firejail --tree | grep ssh-agent
$
$ ls "$SSH_AUTH_SOCK"
ls: cannot access '/tmp/ssh-XXXXXXYk2ZGD/agent.4': No such file or directory

---

As for hanging when trying to daemonize, it seems to be due to the following
issue:

• Firejail breaks daemonizing processes #3491

As a workaround, it looks like it's possible to have the shell put ssh-agent
in the background and redirect its (initial) output in order to read it:

$ firejail /usr/bin/ssh-agent -s >output &
$ . output
Agent pid 4
$ ssh-add -l
The agent has no identities.

It seems to work regardless of --deterministic-shutdown.

Note also that ssh-agent can also be told not to daemonize:

From ssh-agent(1):

     -D      Foreground mode.  When this option is specified, ssh-agent will
             not fork.

Which also works:

$ firejail /usr/bin/ssh-agent -s -D >output &
$ . output
Agent pid 4
$ ssh-add -l
The agent has no identities.

Not sure whether it would be better to use -D or not. Same for
--deterministic-shutdown.

Luckily there are several other SSH
agents that do work
when firejailed.

Good to know.

[glitsj16]

@kmk3 Thanks for researching and the workarounds. After further experimentation I settled on a ssh-agent systemd user service hardened via firejail. For me that's the most convenient way, as I had some trouble to implement your workarounds via shell scripting. In both cases I had to run the . output step manually from CLI for them to work. If you ever find a way to do that from a wrapper script, I would be interested to learn.

Until there's a fix for #3491 it seems there's nothing much to do here for the ssh-agent profile.

[kmk3]

@glitsj16 on Apr 6:

@kmk3 Thanks for researching and the workarounds.

No problem.

I had some trouble to implement your workarounds via shell scripting. In both
cases I had to run the . output step manually from CLI for them to work.
If you ever find a way to do that from a wrapper script, I would be
interested to learn.

Here's an attempt (it's not very pretty, but it works):

myssh-agent (simplified version):

#!/bin/sh

# Put it in a restricted directory since it's intended for eval
output="$HOME/.config/firejail/ssh-agent-$$.tmp"
firejail /usr/bin/ssh-agent -s "$@" >"$output" &
sleep 1
cat "$output"
rm -f "$output"

myssh-agent (with error handling):

#!/bin/sh

# Put it in a restricted directory since it's intended for eval
dir="$HOME/.config/firejail"
mkdir -p "$dir"
output="$dir/ssh-agent-$$.tmp"

firejail /usr/bin/ssh-agent -s "$@" >"$output" &
sleep 1
cat "$output"

# Just in case it fails to write in time
error=0
if ! grep -q 'export SSH_AGENT_PID' "$output"; then
	printf 'error: incomplete output\n' >&2
	error=1
fi

rm -f "$output"
exit "$error"

Run it as:

eval "$(myssh-agent)"

[glitsj16]

Thanks for the shell scripts. Very nice!