To facilitate streaming data from one OpenTSDB instance to another, avoiding intermediate files. If the import "path" is "-", it'll read from STDIN instead.
For example, something like this for migrating between two configs (with different salting or byte width configs etc):
tsdb scan --config=old.conf 1y-ago sum my.metric | tsdb import --skip-errors -- -
There may already be another way to achieve this?
A very quick check of the above got ~70k/s on an anaemic VM backed by a standalone Hbase (~85k unique metrics out of the 999981):
2015-10-06 21:23:42,263 INFO [main] [TextImporter.importFile]: Processed - in 14855 ms, 999981 data points (67316.1 points/s)
2015-10-06 21:23:42,264 INFO [main] [TextImporter.main]: Total: imported 999981 data points in 14.861s (67287.2 points/s)
(i wondered if TextImporter could also leverage aec67bf to improve its speed?)
Allow "tsdb import" to read from STDIN
Hey @manolama - what are the chances of this getting wangled into 2.2? Might help people with migrating data after upgrades...
Hey @manolama - perhaps this can go into 2.3?