In most cases we will find default sshd_config systems, where there won't be many surprises, but edge cases shall arise all the time in the real world, and we should be prepared for them.
Such variety should be implemented with containers and VMs to be heavily tested.
These should be so heavy that they will be infeasible in development time, but a CI system should be able to take care of that and run all these tests periodically.