A set of Pushpop jobs that collect data for anyone to use
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Open Data Collectors

A set of Pushpop jobs that collect data from various APIs over time and push data into Keen IO.

There are 2 ways to use this project:

  • Use the Keen IO Project ID and Read Key specified in each collector's documentation to use the data.
  • Capture the data yourself by cloning and deploying this repository pointing to your own project ID.

Note: Install keen-cli to run example CLI queries.


Bitcoin Prices

Uses the BitcoinAverage API to capture Bitcoin price trends over time.

  • Code – jobs/bitcoin_prices.rb
  • Interval - Every minute
  • Contact Info - @dzello
  • Runs On – Heroku
  • Example Event - Extract One
  • Keen IO Data Set Information
    • Project ID: 53f3a8687d8cb95095000001
    • Read Key: 99a06e48fd7fb1279bc40995160eb0b61a9e0efaab8b651b029f0d895f77c0a804ba089282eff28bf8ad07f337422441d0542b7feaac9fea1e92fc153ee7efc51afad3276bda8d7754a338b349d540bfb402cba0dfdc82498c217054efd8abd0f47a0c0bc963bbdf0dc938c91b17d9f2
    • Collection Name: bitcoin-prices

Example CLI Queries

Note: Make sure to set environment variables in .env or specify --project and --read-key parameters.

# price last 30 minutes piped to spark
$ keen average -c bitcoin-prices -y "USD.averages.last" -t last_30_minutes -i minutely --spark | spark

# highest USD price today
$ keen maximum -c bitcoin-prices -y "USD.averages.last"

# total USD volume last 24 hours
$ keen extraction -c bitcoin-prices --property-names "USD.averages.total_vol" --latest 1
    "USD": {
      "averages": {
        "total_vol": 38276.39


More jobs! More open data sets!

If you add a job please fill out all the details (use Bitcoin Prices as a guide). Keen IO project and credentials are not required if you expect someone to capture the data on their own and not use a shared project.