Support for multiple types.db files in config.toml

[sanjmonkey]

Appears influxDB currently has support for just 1 typesdb file. Those who maintain custom types.db files for their collectd implementations will have to merge types.db's into 1 file if they use "Type" from both.

Reproduce:
[input_plugins.collectd]
enabled = true
port = 25826
database = "collectd"
typesdb = "/usr/share/collectd/types.db.custom"
typesdb = "/usr/share/collectd/types.db"

/opt/influxdb/current/influxdb -config /opt/influxdb/shared/config.toml
Couldn't parse configuration file: /opt/influxdb/shared/config.toml
[11/10/14 15:13:00] [INFO] Loading configuration file /opt/influxdb/shared/config.toml
Near line 57, key 'input_plugins.collectd.typesdb': Key 'input_plugins.collectd.typesdb' has already been defined.

or similarly (using space or , as separator)
[input_plugins.collectd]
enabled = true
port = 25826
database = "collectd"
typesdb = "/usr/share/collectd/types.db.custom" "/usr/share/collectd/types.db"

/opt/influxdb/current/influxdb -config /opt/influxdb/shared/config.toml
Couldn't parse configuration file: /opt/influxdb/shared/config.toml
[11/10/14 15:14:22] [INFO] Loading configuration file /opt/influxdb/shared/config.toml
Near line 56, key 'input_plugins.collectd': Near line 56: Expected a top-level item to end with a new line, comment or EOF, but got '"' instead.

Could we support multiple to align with collectd?

[kimor79]

I think I can add this functionality. Looking at the toml spec though, multiple typesdb files would need to be configured as an array like typesdb = [ "/path/to/types.db", "/path/to/custom/types.db" ] which is incompatible with the current typesdb = "/path/to/types.db". I'll leave it to the InfluxDB owners to decide whether to break incompatibility or add a new config item (e.g., "typesdbs" <- plural)

[rvrignaud]

Maybe influxdb could support the inclusion of every file in a directory (ike typesdb-dir) ?
Directory make usage with configuration management tools way easier.

[everflux]

This would be really awesome to be able to have a collectd/influxdb config consisting of the same set of type definition files. (esp. when managed by puppet, multiple files are easier than a single big file)

[aglagla]

+1

[pires]

I'm on it but can you guys provide me with an example of two of these files so I can add them to tests?

[otoolep]

The directory option would best best, since it would not be a breaking change.

[pires]

@otoolep should we support both configuration entries simultaneously or one overrides the other, e.g. if typesdb is set, ignore typesdb-dirs?

[kimor79]

collectd.conf accepts multiple TypesDB parameters and so the types.db files might not be all in the same directory. Also there could possibly be other files in the default types.db location (I've seen postgres queries and java files before).

[pires]

@kimor79 #4740 implements support for multiple directories and multiple files per directory.

[pires]

Can anyone perform system tests with the PR related to this issue, #4740?

[mengelmann]

@pires how can I help to merge PR #4740 into master? What is required to perform system tests?

[mengelmann]

@otoolep I think that this feature is quite useful, and PR #4740 created by @pires is ready to merge. How can I help to get it merged into master?

[pires]

@mengelmann @otoolep is no longer involved with InfluxDB development. Perhaps @jwilder can help here? Anyway, we were looking for users that could merge my PR, build InfluxDB and test with their deployments to see if everything works as expected.

[mengelmann]

@pires OK, count me in.

We're using on production build with merged PR #6090 (multiple Collectd plugin listeners). I'll add PR #4740 to the build and make tests.