SSH gets disabled on a new Tumbleweed install

[ubenmackin]

See post for more details.

@FroggyFlox found that Tumbleweed has changed the default sshd_config file with regards to SFTP. This causes two SFTP entries to be added to the file, and then ssh won't come up on subsequent boots.

[RlndVt]

I just did a VM install of tumbleweed and I can't connect through SSH.

In my leap VM the rockstor config is installed into the sshd_config file, whilst in the tumbleweed install a sshd_config.d/rockstor-sftp.conf is created.

The issue in this case is that the sshd_config file does not exist. Creating the file with content "PermitRootLogin yes" 'fixes' the SSH config.

Edit:
Oh that's right. Tumbleweed moved most(/all) of it's config files to /usr/etc.

In that case the 'problem' is a discrepency between PermitRootLogin yes on a Leap install (not a fan long term, but important initially to get the keys transferred,) and PermitRootLogin prohibit-password on Tumbleweed.

[RlndVt]

To add, if I attempt to ssh to the account created by Rockstor initialisation, I get the error that the user is not in the AllowUsers list.

Curiously when the /etc/ssh/sshd_config file exists with PermitRootLogin yes then I can login under the non-root account.

[phillxnet]

@RlndVt Thanks for commenting here. However I don't think you are experiencing the same issue. This one relates to our prior sftp config breaking ssh config: we use sshd's subsystem for sftp.

Our installers have 4.5.8-0 (RC5) that is Release Candidate 5 in as we had a blocker that prevented us from creating newer ones across all OS/arch targets. In our first stable version of the rpm (4.6.0-0), which was first released in testing channel as RC7 in testing channel, we adapted to this newer config file arrangement adopted by Tumbeweed.

See the 4.6.0-0 release notes:
https://forum.rockstor.com/t/v4-5-testing-channel-changelog/8546/13
which indicate this was fixed via:

• Account for Tumbleweed sshd config changes #2501 by @phillxnet 1 in #2555

So updating to that version, or newer, should sort this out. There are newer versions than this in both current stable and testing, but our latest testing channel release of 5.0.5-0 is not yet RC status.

I'll close this issue however as:
Fixed by #2555

Given we do now account for TW's variations in sshd config location/override etc.

Once we have a reproducer post last stable, or in testing, we can re-open or create a fresh issue based on our current stable or testing code - as both do things differently now.

Hope that helps, and again apologies for the potential distraction here. Must do some issue pruning for such things as much of our code as changed in the interim on such things.

[RlndVt]

@RlndVt Thanks for commenting here. However I don't think you are experiencing the same issue. This one relates to our prior sftp config breaking ssh config: we use sshd's subsystem for sftp.

I felt that as my issue matches the title of the issue, (but not the body,) it would be best to build on this issue. If you prefer I open a seperate issue I can.

Our installers have 4.5.8-0 (RC5) that is Release Candidate 5 in as we had a blocker that prevented us from creating newer ones across all OS/arch targets. In our first stable version of the rpm (4.6.0-0), which was first released in testing channel as RC7 in testing channel, we adapted to this newer config file arrangement adopted by Tumbeweed.

See the 4.6.0-0 release notes: https://forum.rockstor.com/t/v4-5-testing-channel-changelog/8546/13 which indicate this was fixed via:

* Account for Tumbleweed sshd config changes [#2501](https://github.com/rockstor/rockstor-core/issues/2501) by [@phillxnet 1](https://github.com/phillxnet) in [#2555](https://github.com/rockstor/rockstor-core/pull/2555)

So updating to that version, or newer, should sort this out. There are newer versions than this in both current stable and testing, but our latest testing channel release of 5.0.5-0 is not yet RC status.

I'll close this issue however as: Fixed by #2555

Given we do now account for TW's variations in sshd config location/override etc.

Once we have a reproducer post last stable, or in testing, we can re-open or create a fresh issue based on our current stable or testing code - as both do things differently now.

Hope that helps, and again apologies for the potential distraction here. Must do some issue pruning for such things as much of our code as changed in the interim on such things.

@phillxnet
Do I understand correctly that I should be able to SSH in, (as root, with a password) using a clean Rockstor-TW install, zypper dup'd, that is updated to 5.0.5-0?

For in my experience I cannot.

[FroggyFlox]

@RlndVt, thanks a lot for the follow-up here and the clarification. I can confirm that I experience the same as you.

1. Install latest Rockstor TW ISO (4.5.8-0)
2. Distribution update: zypper dup --download-in-advance
3. Reboot
4. Go to Rockstor webUI, activate Testing channel, and update to Rockstor 5.0.5-0
5. Reboot
6. Try to ssh from client --> refused

/etc/ssh/sshd_config.d/rockstor-sftp.conf is present and includes:

Subsystem    sftp    internal-sftp
AllowUsers root

It seems that AllowUsers root no longer takes precedence on PermitRootLogin yes as you observed in #2501/#2555.
Adding PermitRootLogin yes to /etc/ssh/sshd_config.d/rockstor-sftp.conf and restarting the sshd service restores ssh access for root.
Note that as indicated by @RlndVt, the os default in /usr/etc/ssh/sshd_config is now PermitRootLogin prohibit-password. I'm not if that changed from when you worked on #2555/#2501, though.

[RlndVt]

By adding AllowUsers root, we are also limiting SSH to only the root user.

[phillxnet]

@RlndVt My apologies, I should first have requested your rockstor version, and the path that lead to, i.e. the reproducer.

@phillxnet Do I understand correctly that I should be able to SSH in, (as root, with a password) using a clean Rockstor-TW install, zypper dup'd, that is updated to 5.0.5-0?

For in my experience I cannot.

You do. And now I do also :), so many thanks for your persistence with this. And again this issue is old and relates to the specific, and believed to be resolved:

This causes two SFTP entries to be added to the file ...

plus this mechanism has now all been re-done in the referenced:

• Account for Tumbleweed sshd config changes Account for TW sshd config changes #2501 by @phillxnet 1 in Account for Tumbleweed sshd config changes #2501 #2555

So we should have closed it back then. But there was a more detailed one which should have linked to this. So lets leave this closed, at least for now.

I felt that as my issue matches the title of the issue, (but not the body,) it would be best to build on this issue.

Yes, the mistake here is our leaving this open. Again my apologies. We just need more time/folks to review open issues and close those that are too vague - without exact reproducers etc. The mistake/failure here was in our procedures/available (human) resources.

If you prefer I open a seperate issue I can.

And Yes, that would be great thanks, as we then have the all-important attribution credit :). I think @FroggyFlox's comment re:

Note that as indicated by @RlndVt, the os default in /usr/etc/ssh/sshd_config is now PermitRootLogin prohibit-password. I'm not if that changed from when you worked on #2555/#2501, though.

Could be the culprit. Lets use the new issue to pin this down.

I strongly suspect this could account for things. We have some upstream (in TW) change in behaviour that we need to track down as it seems we have to yet-again, adapt to TW's ever changing goal-posts. And a specific issue with detailed reproducer can serve to pin this down. After the last adaptation all was well (at the time), but alas we must adapt again. We can pop this new issue on our next Stable milestone I think as although our TW installer is currently labelled as:

"Development/Advanced-user/Rescue use only"

It is in our interests re Leap's transition to an as yet undefined successor, we are likely looking at a single Slowroll upstream in time anyway - with TW being it's upstream !

By adding AllowUsers root, we are also limiting SSH to only the root user.

Yes, that is intended behaviour. Here we are after restoring that behaviour: i.e. root ssh login, and only user specific login with sftp addition via Web-UI: and then limited to a chroot. Which was disabled upstream (but not on all architectures/image types) within TW only: as detailed in the linked issue and corresponding PR. A complication is our tie-in re the sftp upstream ssh module, not the one used by openSUSE.

@FroggyFlox Thanks for the invaluable follow-up here. Much appreciated.

Note that as indicated by @RlndVt, the os default in /usr/etc/ssh/sshd_config is now PermitRootLogin prohibit-password. I'm not if that changed from when you worked on #2555/#2501, though.

I don't remember this, and what we had did work, but as-is too often the case with TW, not for that long!!

Thanks folks. And we must get our messy issue backlog in order, should be easier once we transition master to our next stable based on all the updates now in place within testing, all in good time hopefully. I think we even have some CentOS specific issue laying around!!