While we're refactoring the build system and all, maybe we should go to a "src layout"
https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/
I find the src layout to be more robust and less prone to error.
I'm happy to do a PR if y'all think it's a good idea.