Configuration is confusing and incomplete

[codepope]

The current use of config.yaml means that only two parameters can be passed to a node from the yaml file and any other parameters have to be passed within the JavaScript application.

On the understanding that the idea of a config file is to allow the pipeline configuration in the JavaScript to be practically immutable and any changes in setup should be made in the config.yaml, then the current setup fails that goal.

Consider the mongodb adapter. It requires the uri and namespace parameters to be set. The uri is set in config.yaml but the namespace is set in the application.js.

A stable application JavaScript and a adaptable and flexible config.yaml is desirable, for resiliance in production and consistency in documentation.

[nstott]

The part that bothers me most about this is that the database configurations are somewhat magical. these aren't documented anywhere other then in the adaptor code.

Adaptors can take different arguments, eg the file adaptor takes a uri, whereas the mongo adaptor takes a uri, namespace, and a couple of boolean flags. these options need to be exposed to the end user

[codepope]

To follow on from that issue.

My personal preference would be to move all adapter config to config.yaml...

1. Ensure adapters register their required parameters at startup

2. For the config.go element to use this registered data.

3. For the config.yaml settings to be over-ridable during construction of sources/sinks in the JavaScript

4. Add "transporter about" (lists adapters loaded) and "transporter about adaptername" lists parameters available for the named adapter.(addressing some of @nstott 's concetn)

Optional
5) Consider adding support for named transformers with parameters so they are homogenous with sources and sinks.

[nstott]

I'm splitting the item '4' out into it's own issue ( #21 ). I'm uncertain about the rest of these, but I think that the adaptor config can be dealt with easily.