[SELinux] TOTP don't seems to work

[doubleailes]

I set TOTP on my account.

But after the sucessfull registration, all my verification are refused.

Is there any way to get my account back?
With the scratch code?

[speed47]

The scratch codes can help if for some reason you no longer have the TOTP app or it no longer works correctly. Each code can only be used once, however.

To disable TOTP for your account you can use the --osh selfMFAResetTOTP command. It requires the TOTP of course, but you might want to try a scratch code there.

Once you've unlocked yourself, to be able to help you better, please describe the steps you followed to enable TOTP on your account.

[doubleailes]

Ok i tried the --osh selfMFAResetTOTP and insert my scratch code but i did not work
Of course i tried a couple ones in the list.

I'm running Centos8 in VM. My timedate seems update.

I used the command --osh selfMFASetupTOTP

and get the result:

Warning: pasting the following URL into your browser exposes the OTP secret to Google:
*****
Failed to use libqrencode to show QR code visually for scanning.
Consider typing the OTP secret into your app manually.
Your new secret key is: ****..***
Enter code from app (-1 to skip): ******
Code confirmed
Your emergency scratch codes are:
  *******
  *******
  *******
  *******
  *******

[speed47]

Did you install pamtester?
Can you give a screenshot of the error you get when trying to login with the TOTP?

[doubleailes]

nope i did install pamtester should it be install on the bastion?

here is the login exemple:

$ bssh --osh selfMFAResetTOTP
*------------------------------------------------------------------------------*
|THIS IS A PRIVATE COMPUTER SYSTEM, UNAUTHORIZED ACCESS IS STRICTLY PROHIBITED.|
|ALL CONNECTIONS ARE LOGGED. IF YOU ARE NOT AUTHORIZED, DISCONNECT NOW.        |
*------------------------------------------------------------------------------*
Enter passphrase for key '/home/******/.ssh/id_ed25519': 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
Multi-Factor Authentication enabled, an additional authentication factor is required (OTP).
Verification code: 
***@***.***.***.***: Permission denied (keyboard-interactive).

[doubleailes]

I just checked. pamtester is install into my bastion.

[speed47]

There might be something wrong on the configuration of the google_authenticator PAM module on your centos machine. When there is something wrong in its config, it'll always deny authentication. Centos 8 is part of the OSes that are automatically tested on each release, including for MFA, so it should work if the configuration is ok and the proper packages are installed.

If there is the "debug" keyword on the google_authenticator pam module line in /etc/pam.d/sshd file, you should get some diagnostic information in the system logs, that should give a hint about the problem.

[doubleailes]

There is no real equivalent of system logs in CentOs8 can you be more specific in which log it will be store.
the only line talking about google_authenticator is:
auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so secret=~/.otp

I guess debug should be add here:
auth [success=ok new_authok_reqd=ok ignore=ignore default=bad module_unknow=ignore] pam_google_authenticator.so debug secret=~/.otp

[speed47]

Just booted a brand new CentOS 8 VM to be sure, and I can indeed get OTP to work by following the documentation (just wanted to be sure).

You are right, the "debug" keyword is to be added to the line you specified.

When you've done this, your PAM will log using your system's syslog, under the "debug" level. Here's what I get with mine:

Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Accepted key ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY found at /home/joe/.ssh/authorized_keys2:1
Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Partial publickey for joe from 127.0.0.1 port 46634 ssh2: ED25519 SHA256:r0e/w4HzCzPArzE3bbqu3fSI7Sko29ODO7yjNFt08QY
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: start of google_authenticator for "joe"
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: Secret file permissions are 0400. Allowed permissions are 0600
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" read
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: shared secret in "/home/joe/.otp" processed
Oct 31 22:01:54 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: google_authenticator for host "127.0.0.1"
Oct 31 22:01:54 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive for joe from 127.0.0.1 port 46634 ssh2 [preauth]
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: no scratch code used from "/home/joe/.otp"
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: Accepted google_authenticator for joe
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: "/home/joe/.otp" written
Oct 31 22:02:08 c1bfbf5957bc sshd(pam_google_authenticator)[2242]: debug: end of google_authenticator for "joe". Result: Success
Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Postponed keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2 [preauth]
Oct 31 22:02:08 c1bfbf5957bc sshd[2239]: Accepted keyboard-interactive/pam for joe from 127.0.0.1 port 46634 ssh2

The location of the file will depend on your system's syslog configuration.
On my test system I've just put up, I installed syslog-ng, and added these 2 configuration lines to get all the logs into the same file:

destination d_all { file("/var/log/all.log"); };
log { source(s_sys); destination(d_all); };

(at the end of /etc/syslog-ng/syslog-ng.conf)

[doubleailes]

ok it's a permission issue.

[speed47]

Indeed, it seems your user can't write to its own home directory. I suppose you fixed it successfully?

[doubleailes]

Nope. I change folder permission other than 700 the TOTP check is refuse.

[doubleailes]

How can i be sure the pam process is running under my user?

[speed47]

How can i be sure the pam process is running under my user?

It's running under your user when pamtester is called, which is what the bastion does:

the-bastion/bin/shell/osh.pl

Line 878 in b0eaf15

my $pamsysret = system('pamtester', 'sshd', $sysself, 'authenticate');

You can try it manually this way, running it directly on your centos machine under your user:

su - yourusername -s /bin/bash
pamtester sshd yourusername authenticate

[doubleailes]

You can try it manually this way, running it directly on your centos machine under your user:

su - yourusername -s /bin/bash
pamtester sshd yourusername authenticate

It worked.
So i checked my home user folder is 700 and my .opt is 600

[doubleailes]

Did change selinux on your centos8 VM?

[doubleailes]

I changed selinux to permissive and it worked.

[speed47]

Ah! Interesting.

This isn't catched during the tests because we use Docker to test on several distro flavors, so of course SELinux doesn't apply there.

We never stumbled upon that because we use Debian in production. Falling back to permissive is not acceptable for a security asset, so I think we'll have to take some time to write a proper SELinux policy.

[doubleailes]

Thank for all your help.
It's really a great tool.

I'll be happy to put it in production when the policy will be written.

[speed47]

pam, called by sshd, needs to be able to write the otp files. The following policy seems to be sufficient with our tests:

cat >the-bastion.te <<EOF
module the-bastion 1.0;

require {
	type var_t;
	type sshd_t;
	type user_home_t;
	type user_home_dir_t;
	class file { create getattr rename setattr unlink open read write };
}

# needed for user TOTP (~/.totp and ~/.totp~XXXXXX temporary file)
allow sshd_t user_home_dir_t:file { create getattr rename setattr unlink open read write };
allow sshd_t user_home_t:file     unlink;
# needed for root TOTP (/var/otp/root and /var/otp/root~XXXXXX temporary file)
allow sshd_t var_t:file           { create getattr rename setattr unlink open read write };
EOF

checkmodule -M -m -o the-bastion.mod the-bastion.te
semodule_package -o the-bastion.pp -m the-bastion.mod
semodule -i the-bastion.pp