Where should we put example interfaces to simulation packages?

[namurphy]

While discussing PlasmaPy top-level package structure during our community meeting today and at the APS DPP meeting last week (PlasmaPy/PlasmaPy-PLEPs#26), the issue came up of where to put example interfaces to existing simulation packages.

For some background, our current plan is to create classes in plasmapy.simulation that can be used to describe the initial conditions, boundary conditions, and computational domain independently of the numerics. We also want to create an abstract interface that can be implemented with different plasma simulation codes (including the ones we develop along with codes from the community).

It would be really helpful to the community for us to have examples for how to build implementations for these interfaces for different codes that are already in existence. We weren't able to reach a consensus on where to put the example interfaces, so we're raising this issue. Where would be the best place to put these example interfaces to plasma simulation codes that have already been developed by members of the broader plasma physics community?

[namurphy]

SunPy's practice has been to include general functionality in their core package, while functionality specific to a particular instrument gets put into an affiliated package for that instrument. The affiliated packages use the functionality within the core package. This separation makes the core repository cleaner and more maintainable, and allows instrument teams to have control over the corresponding instrument packages.

My preference would be to take a similar approach for PlasmaPy, and not implement interfaces to external plasma simulation codes in the core package. My preference would be to have implementations to these interfaces in affiliated packages.

[StanczakDominik]

@rocco8773 raised the very interesting idea of having a plugin system on Riot. This is something that may make a lot of sense - I'll dig into it and report back.

[rocco8773]

We need to do some playing around, but I'm thinking native namespace packages is the way to go. PEP 420 covers the various types of namespace packages and Python has a sample github repository here.

I think we could have a top-level namespace sub-package plasmapy.extensions, plasmapy.addons, or the like. We could also have a namespace package within simulation like plasmapy.simulation.interfaces that would focus on simulation interfaces with existing simulation codes. This would allow us to have separately install-able plasmapy-addons packages that would not ship with plasmapy but once installed would be accessible within the plasmapy framework. (That's cool!!!)

The plus side...

1. This would allow us to easily implement simulation interfaces and/or instrument specific analysis without it bloating plasmapy.
2. All these features could be included in separate repositories that are maintained by separate coordinating groups than the plasmapy main coordinators.
3. They would only be installed by users that want them.
4. Would allow users to develop their own "addons" that fall outside PlasmaPy's current scope.

The downside, maybe upside...

1. We lose control of enforcing these namespace packages.
2. Anyone could create a plasmapy-dosomething package and have it registered on PyPI.
3. If the package owner does not properly maintain that package, then any other user with the "addon" package installed would have their plasmapy.addons sub-package broken.

[namurphy]

We need to do some playing around, but I'm thinking native namespace packages is the way to go.

Agreed! This is one of the best possibilities among the ones that we've discussed, I think. Thanks for suggesting it!

[namurphy]

Speaking of which, I should probably get back to working on this simulation PLEP that's been on hold for a while now... Whatever we decide on for this issue should probably get incorporated into that.

[StanczakDominik]

I've finally sat down to read up on this, and if I'm understanding this right, for now focusing only on the simulation interafaces, what you're proposing is essentially:

1. add a plasmapy.simulation.interfaces subpackage that has no __init__.py file, and, in fact, does very little in the main repository; add it explicitly under the packages key in setup.py
2. as an example: create a plasmapy/plasmapy-SIMULATIONCODEONE repository that includes, say, a module named plasmapy.simulation.interfaces.simulationcodeone implementing a single-file interface for this simulation code (or even a dedicated subpackage if the interface becomes complex enough to warrant splitting into modules)
3. as another example: create a plasmapy/plasmapy-SIMULATIONCODETWO repository that includes a module named plasmapy.simulation.interfaces.simulationcodetwo
4. if you pip install plasmapy plasmapy-simulationcodeone plasmapy-simulationcodetwo, you can import plasmapy.simulation.interfaces.simulationcodeone and simulationcodetwo

[StanczakDominik]

I think I like this idea. CI for the side packages could be accomplished by pip installing plasmapy, then repo itself, and we'd get quick tests for those. The test infrastructure can then mostly be copied over.