Plugin system

[Luthaf]

I have been thinking about this for a while, here is a request for comment (RFC) to shape it a bit more and gather feedback. The implementation of this will take time and is not high priority, but I really want to get this right.

Motivations

A nice way for the code to be easy to extend would be to provide a plugin system. A plugin is some piece of code that is not part of the main lumol code, and is not compiled at the same time than the main binary, nor linked to it.

Having a plugin system would mean that people can write their own code and use it with lumol without recompiling the whole project. It is highly desirable that this plugin system works easily with the input files too. I'd like the loading/use of a plugin to be as easy as:

[input]
version = 1

[plugins]
foo = "path/to/foo/plugin.so"

[[pairs]]
atoms = ["A", "A"]
foo = {} # potential foo is defined inside the foo plugin

[[pairs]]
atoms = ["A", "B"]
bar = {} # potential bar is also defined inside the foo plugin

Types of plugins

I see three main kinds of plugins for the Lumol use case:

1. Compiled plugins written in Rust;
2. Compiled plugins written in C (or any C-exporting language: Fortran, ...);
3. Interpreted plugins written in Python.

(1) and (2) are fast, (3) is nice for writing quick prototypes. (1) is easier to implement, and (2) allow to call into other codes for potential computations (to use DFT level energy for example). (1) and (2) would be based on shared library and dynamic loading, while (3) will use some kind of future python binding.

Plugins should be usable to extend lumol in all the possibles means: not only writing new potentials, but also new integrators, new Monte-Carlo moves, ... Everything that is defined as a trait for extensibility in the code should be available as a plugin

Proposed implementation strategy

I will focus on Rust plugins for now, as they are way easier to implement, and the C plugins can be built on the top of the Rust plugins mechanisms.

Every plugin would be compiled as a shared library, and expose at least a single function: lumol_register, taking a single argument of type &mut PluginRegistry defined in lumol_input crate. This function returns a Result in case something goes wrong. The PluginRegistry will define a few method to register extension points

extern crate lumol_input;
use lumol_input::{PluginRegistry, PluginError};

struct Foo;
struct Bar;

impl PairPotential for Foo {...}
impl PairPotential for Bar {...}

pub fn lumol_register(reg: &mut PluginRegistry) -> Result<(), PluginError> {
    try!(reg.add_pair::<Foo>("foo"));
    try!(reg.add_pair::<Bar>("bar"));
    Ok(())
}

// add_pair is defined in lumol_input as
impl PluginRegistry {
    pub fn add_pair<T: PairPotential>(&mut self, name: &str) {...}
   // a given potential name can only be given once
}

Reading an input file will follow this new algorithm:

• Create a new PluginRegistry;
• Register the standard potentials in it;
• Parse the input file;
• Load the plugins;
• Call lumol_register for all the plugins using the PluginRegistry;
• Read the input file, and at every section ask the PluginRegistry if there is any potential/integrator/MC move with the given name, and use it.

Extra goodies

This architecture would allow for more modularity. The lumol_core crate could only contains the main structs (UnitCell, System, Simulation, ...) and the traits (Propagator, Potential, ...). Standard implementation would then live in another crate (LennardJones, MolecularDynamics, ...), that is use by lumol_input.

The problem with this is that traits like MCMove would live in a separated crate than the MonteCarlo structure.

Unresolved questions

As rustc does not guarantee a stable ABI for now, calling lumol_register with two different rustc version could break everything. This may be solved by using a C ABI, but I do not know if it supports passing and returning Rust types.

---

Thank you for reading this, now please comment if you have any feedback or if clarifications are needed!