You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi
First up - great work on the library, looks like it is going to solve a bunch of our problems so we dont have to! 馃憤
I'd like to make it easy to create source 'plugins' which can lie outside the the joerd code tree so that users can easily get it to work with their own sources. Given that a huge portion of the issues are requests for supporting other data sources, I think it could be good to make it easy to extend joerd at runtime.
Proposed changes:
Allow plugin in plugin.py to accept a full path to a module to import, rather than assuming it lies in the joerd package. This would in turn require _sources in server.py to support a different way of specifying modules to allow external modules. Something like:
forsourceincfg.sources:
if'module'insource:
#import module from outside joerd packagecreate_fn=external_plugin(source['module']) # could also adjust the signature of 'plugin' to support different arugments, rather than creating a new plugin functionelse:
source_type=source['type']
create_fn=plugin('source', source_type, 'create')
Define an interface for source plugins to implement.
As far as I can see, joerd implicitly defines an interface - from the _sources function it appears a source must support a create method. Other methods to support are listed here: https://github.com/tilezen/joerd/blob/master/CONTRIBUTING.md
It could be good to explicitly define the interface - e.g. provide an abstract base class for plugin developers to implement.
I'd also advocate moving the documentation on the interface from CONTRIBUTING to the README or some dedicated documentation.
The line create_fn = plugin('source', source_type, 'create') seems slightly counter-productive as plugin supports a custom entry point function, but server then hardcodes it to create. I'd either take the name of the entrypoint from the configuration (e.g. sources['entrypoint']), or just define that all plugins must implement create and attempt to call it (throwing an appropriate error if it doesnt exist).
I think these are relatively small changes which would allow users to implement their own data sources without having to either
a) fork the entire codebase
or
b) wait for their pull request to be merged and a new release to be created.
Popular sources could still be merged into the base library, or even a separate plugin library maintained?
Same approach could be applied to stores and outputs
The text was updated successfully, but these errors were encountered:
We've been working on v2 which addresses some of these concerns over at https://github.com/mojodna/marblecutter/. Please poke around over there to see if it suits your needs.
Hi
First up - great work on the library, looks like it is going to solve a bunch of our problems so we dont have to! 馃憤
I'd like to make it easy to create source 'plugins' which can lie outside the the joerd code tree so that users can easily get it to work with their own sources. Given that a huge portion of the issues are requests for supporting other data sources, I think it could be good to make it easy to extend joerd at runtime.
Proposed changes:
plugin
inplugin.py
to accept a full path to a module to import, rather than assuming it lies in thejoerd
package. This would in turn require_sources
inserver.py
to support a different way of specifying modules to allow external modules. Something like:As far as I can see,
joerd
implicitly defines an interface - from the_sources
function it appears a source must support acreate
method. Other methods to support are listed here:https://github.com/tilezen/joerd/blob/master/CONTRIBUTING.md
It could be good to explicitly define the interface - e.g. provide an abstract base class for plugin developers to implement.
I'd also advocate moving the documentation on the interface from CONTRIBUTING to the README or some dedicated documentation.
The line
create_fn = plugin('source', source_type, 'create')
seems slightly counter-productive asplugin
supports a custom entry point function, butserver
then hardcodes it tocreate
. I'd either take the name of the entrypoint from the configuration (e.g.sources['entrypoint']
), or just define that all plugins must implementcreate
and attempt to call it (throwing an appropriate error if it doesnt exist).I think these are relatively small changes which would allow users to implement their own data sources without having to either
a) fork the entire codebase
or
b) wait for their pull request to be merged and a new release to be created.
Popular sources could still be merged into the base library, or even a separate plugin library maintained?
Same approach could be applied to stores and outputs
The text was updated successfully, but these errors were encountered: