k-openaq

A Krawler based service to download data from OpenAQ

Description

The k-openaq allows to download OpenAQ data for a given list of countries. The available countries is available here. Also, a full Json output can be retrieved using the the following query

The available variables are:

• pm25: particulate matter PM255
• pm10: Particulate matter PM10
• so2: Sulphur dioxide
• no2: Nitrogen dioxide
• o3: Ozone
• co: Carbon monoxide
• bc: Black carbon

The job relies on the OpenAQ API v2.0.

Note

Because of Rate limiting It is recomended to register to OpenAQ to get an API Key.

The following diagram illustrates how the job works:

Configuration

The job can be configured by setting the following environment variables:

Parameter	Description
API_KEY	the API Key.
COUNTRIES_IDS	an array to specify the countries to take into account. By default: '2', '134','133','132'
VARIABLES	an array to specify the variables to scrape. By default: 'pm25', 'pm10', 'so2', 'no2', 'o3', 'co', 'bc'
QUERY_LIMIT	the limit of returned results. By default: 1000
TTL	the time to live of the data. By default: 7 day
TIMEOUT	the timeout of the request. By default: 60 * 60 * 1000
DB_URL	the database url. By default: mongodb://localhost:27017/openaq

Deployment

We personally use Kargo to deploy the service.

Contributing

Please refer to contribution section for more details.

Authors

This project is sponsored by

License

This project is licensed under the MIT License - see the license file for details