Move non-user-facing packages to internal?

[rakyll]

Should the packages such as experimental and tests be moved to the internal package, so the top level of the package is not populated by the non-user-facing-packages on godoc?

[maruel]

I did work on reducing the amount of top level files and directory and I agree it's worth looking at the remaining ones.

experimental is meant to be usable by users at their own risk. So I wouldn't want it to be in an internal package.

tests is not user facing and I agree it'd be nice to be able to hide that but I don't know. It could be under host/tests since the only test there related to host. I'm unsure if it's a good idea though; I kinda like to have all the smoke tests potentially at the same place. On the other hand if we were to use internal/tests then it's true that it wouldn't show up on godoc, then that's a valid point. I'm fine with this idea.

[rakyll]

internal/tests then it's true that it wouldn't show up on godoc

internal packages are showing up on godoc but cannot be imported by users. It is a short way to say, there is nothing you should care about under this directory.

[tve]

I've been lobbying Marc to move things out of internal because it tends to make out-of-tree development more painful. If putting tests under internal means that one can't make use of common test helper code when doing development out-of-tree then I think there's a loss.

[quinte17]

what do you mean by out of tree development?

• someone who just writes a driver and therefore uses pio
• or someone who wants to work on pio?

in any case, everybody is free to fork and/or clone into needed directories.

[tve]

By out-of-tree I mean being able to develop a driver or host component outside of the pio directory tree, i.e. without modifying pio itself. This means that you can't use internal packages. Typically test code consists not only of the tests themselves but also of helpers that make it easier to write tests and if these helpers are in internal packages together with the tests then they can't be used from outside of the pio tree to write tests. See also https://github.com/google/pio/tree/master/doc/apps#using-out-of-tree-drivers

[maruel]

I agree with @tve that enabling out-of-tree drivers is a core feature, and having special functionality for in-tree drivers that is not available to out-of-tree drivers goes directly against that. Extensibility is a core feature. Requiring to fork to add a new driver is explicitly an anti-goal. It's a way of getting code merged due to the way git works but not a way to develop drivers. (People are free to do whatever they want, the idea is that we shouldn't force them to fork in the first place)

I feel internal should normally used for packages, not executables, as executables can't be imported anyway. So putting executables under an internal package is semantically odd (?)

Since tests/ only contain smoke tests, which are all executables, it's probably better to not do this.

Since this removes the last action item, I'll close this issue but I've been discussing with @tve about where smoke tests actually belong. This is an open question but a separate one from the premise of this issue.