Sorry responding so late, we're at small team at Phusion and it's been way too busy the past few months. I've finally come around to investigating docker exec.
I've found that docker exec is just a thin wrapper around nsenter, though with a few changes. Most of the usual cons of nsenter still applies. Therefore, I think it's not a good idea to remove SSH entirely in favor of docker exec. What we can do is replacing nsenter with docker exec, and disabling SSH by default. I've begun work on this.