The latest state of the 52North UoM branch of the data transloader is migrated to https://cwincloud.cc.umanitoba.ca/ccadi/transloader (a private repository!) and will be maintained there. Hence, this public fork of the data-transloader by 52N will be archived.
This tool downloads weather station data from multiple sources and uploads it to an OGC SensorThings API service. This makes the data available using an open and interoperable API.
This tool requires Ruby 2.6 or newer. Once Ruby is installed, you will need to install Bundler to install the library dependencies.
$ cd data-transloader
$ gem install bundler
$ bundle install
This tool should be runnable on Linux, MacOS, and Windows.
The tool can also be ran inside a Docker container, without having to install Ruby at all. See the Docker Instructions for details.
This tool is meant to be scriptable using cron, a scheduling daemon, to automate imports on a schedule. This is done by passing in command line arguments to the tool that handle the sensor metadata and observation download/upload.
The tool is separated into downloading/uploading metadata and downloading/uploading observation data. This is because metadata is not often updated from the source, and the metadata may need manual correction by the data transloader user. The observation data is constantly updated, so it needs to be efficient and avoid checking/creating entities for metadata every time it is run.
Currently supported weather station sources:
- Data Garrison
- Environment Canada
- Campbell Scientific
- Historical weather and energy usage data from the Arctic Institute of North America's research stations
Different source data providers have moderately different implementations and usage. Please see the detailed documentation for an explanation of the source-to-SensorThings API mappings.
- Campbell Scientific Weather Stations
- Data Garrison Weather Stations
- Environment Canada Weather Stations
- Kluane Lake Research Station Historical Energy Usage Data
- Kluane Lake Research Station Historical Weather Data
This tool can be modified and extended by editing its source files. In this environment, you must tell Bundler to also install the "test" group gems:
$ bundle install --with test
James Badger (james@jamesbadger.ca)
GNU General Public License version 3