App for pushing data to ekosystem.slovensko.digital
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
app
bin
config
db
lib
log
public
spec
vendor
.gitignore
.rspec
.ruby-version
Gemfile
Gemfile.lock
LICENSE
Procfile
README.md
Rakefile
app.json
clock.rb
config.ru
docker-compose.yml

README.md

Harvester

Harvester is an open-source application that downloads data from various public sources and pushes data to ekosystem.slovensko.digital.

CircleCI Gemnasium Code Climate Test Coverage

Supported data sources

Currently, Harvester supports the following data-sources:

Currently, there are data sources in ekosystem.slovensko.digital which are not ingested by Harvester. The corresponding code hasn't been open-sourced yet.

Development

Harvester is a Ruby-on-Rails application. However, there is no web process present. The application runs as a backend worker/scheduler and uses the Rails framework to enforce coding rules for the project.

To get the Harvester dependencies running (db, etc), we have prepared a Docker Compose file for you. You can start the dependencies with docker-compose up.

To run and test the application while developing:

  • Prepare the environment with bin/setup
  • Ensure that all tests pass: rake spec
  • Start the worker with foreman start worker
  • Run a selected job with rake
  • Check that the corresponding database tables have been filled in

Contributing

Feel free to contribute other datasets / scrapers or improvements. See issues if you want to help, request a feature or file a bug. Join us on our public slack channel if you want to talk live.