New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
wish: make config file format support pluggable. #181
Comments
To expand a bit, I'd like to see pluggable config sources. Node-config only supports file type config sources, and if we generalize config sources to include asynchronous sources, the plugins could be written for different file extensions, http sources, db sources, etc. Supporting async sources will be a little work, but will remove the biggest restriction with node-config - that it's file based only. |
My recommendation is outsource the file format support to |
Good plan. Do you think this refactor can be added into the 1.x code stream? If it doesn't introduce incompatibilities, then we don't have to wait until the 2.x release. |
It can be tried on a branch and we'll find out. There are perhaps two levels here, one that could be now and the other later:
That is: You could declare your own compilers or loaders for any extension, including ones that already have a default loader. I haven't worked out the details of the interface for that yet. |
@lorenwest, currently we load the various file formats we support in a specific order: var extNames = ['js', 'json', 'json5', 'coffee', 'iced', 'yaml', 'yml', 'cson', 'properties']; ...but we don't document what this order is and it should therefore not be depended on. In our common use-case, a file has only one of the extensions and no siblings with the exact same name but a different extension, so the load order doesn't matter anyway. As we move to making the file format support, it would be ideal to remove this hardcoded ordered list of extensions from our source code. Would you be OK with moving to trying the extensions we support in alphabetical order? Given that the order doesn't matter for the common use-case, I think this is a reasonable trade-off for the flexibility. Alphabetical order should remain fairly predictable if it ever happens to matter. While new extensions may be added later, the alphabetical order between any two pre-existing extensions would remain unchanged. |
I looked more tonight at possibly using the I also found that |
Having a consistent and determinant load order between extensions makes sense. That change would be part of a 2.x release because it could unwittingly break existing installs. It appears that interpret/rechoir aren't libraries that do the loading. Interpret is a database of file extensions to interpreters, and rechoir is a library that load the interpreters so the files can be loaded using Most installations wouldn't need more than one or two file formats. It'd be nice to have a bunch of config modules With a plugin protocol we could offer comment filtering as an option, and plugin authors wouldn't need permission to write and publish a parser for their favorite syntax. |
Came here when looking for the extension priority. If you are suggesting users to not create more than one file ext. under the same name, I suggest putting this in the document. |
config-parser module may have the following structure:
var configParser = require('config-parser');
//Register custom plugin
configParser.registerPlugin(['toml', 'tml'], function(input) { ... return output });
var config = configParser.parse(file); //string/buffer/vinyl |
@markstos I gave it a try in a separate library (to allow myself to start with a clean slate and try different approaches), and I came up with something that should be powerful and flexible enough for different use cases. This how it looks: var configly = require('configly');
configly.PARSERS = {
ini : ini.parse,
// have it as a wrapper to prevent extra arguments leaking
cson : function(str) { return cson.parse(str); },
yml : function(str) { return yaml.safeLoad(str); },
// same options as used within `config` module
properties: function(str) { return properties.parse(str, {namespaces: true, variables: true, sections: true}); },
// use json5 instead of `JSON.parse`
json : json5.parse
// keep the original one
js : configly.PARSERS.js,
};
var configObj = configly(); It is still a singleton, but the module exports load function, |
I see the suggestion of tamtakoe looks good, simple and easy. Why don't just go with it? Currently I have an issue that I want to have a custom type with js-yaml, but since you have hard-coded the .load call so I cannot inject my custom type. I tried to wrap it with global.Yaml but you declared |
@lorenwest See the proposal for pluggable config parsing above. Also note that it came up again a couple of hours ago when someone wanted to use a different algorithm to parse YAML. Here's a related design option. config/parsers.jsCheck for a new config file, The presence if of a known extension there will override the default parsers and the presence of unknown extensions will allow new ones to parsed without support in the core. // config/parsers.js
module.exports = {
ini : ini.parse,
// have it as a wrapper to prevent extra arguments leaking
cson : function(str) { return cson.parse(str); },
yml : function(str) { return yaml.safeLoad(str); },
// same options as used within `config` module
properties: function(str) { return properties.parse(str, {namespaces: true, variables: true, sections: true}); },
// use json5 instead of `JSON.parse`
json : json5.parse
}; Alternatives considered
Using another simple configuration file with object overrides seems to fit very naturally into how the rest of node-config works. It also allows you to configure the parsers in a single place and not have any code in every module where you load |
This is a perfect balance of simplicity and flexibility. I support this change. |
Great idea @markstos |
Thanks for the prompt feedback. I'm adding the "PR Welcome" tag. Perhaps one of the issue followers here or @stalniy from a related ticket will be interested in looking at it. |
@markstos I like the proposal, except that instead of using a convention (predefined filename) I suggest we make it into an env-var pointing to the parser's extension (relative or absolute). It's a pretty straightforward solution, I can implement it if it's a priority. |
#181 - Support custom Parsers definitions from an external source
@iMoses Thanks again for implementing a feature I wanted several years ago. :) |
Supporting additional file formats should be easy.
The first step for this task is to decide on the design to use.
Possible designs
gulp
uses.node-config
currently has no dependencies, so this would be a change in this regard.The text was updated successfully, but these errors were encountered: