(Always a Work In Progress)
2. Who started OpenAQ and what motivated them to do it?
3. Who is part of OpenAQ and what have they built already?
4. What type of organization is OpenAQ and where is it located?
6. Why does OpenAQ require funding, anyway?
8. Do you have a Community Code of Conduct?
9. What kinds of data are stored to the OpenAQ Platform? Where is Air Quality Index data?
10. What are low-cost sensors and why have they been added to the platform?
11. How are data attributed to their sources? How can I find out the sources of the aggregated data?
12. How are the data licensed?
14. What guarantees of the data accuracy are made of the data?
15. What are the ways I can access data?
16. Why can I only access data for the last two years? Where's the rest of the data?
17. How do I cite the OpenAQ platform in peer-reviewed literature?
18. Why is the OpenAQ Platform open-source?
19. Where can I find the open-source software?
20. What is the license of the software?
21. What are some ongoing open-source projects involving OpenAQ?
22. How can I help contribute new data sources? What information is needed for a new data source?
23. I suggested a new data source, but I haven't seen it added yet. What gives?
Contributing Development Skills for Enhancing the Platform
24. What skills are needed to contribute to the code-base?
25. Where can I find existing issues I can help out with?
26. What community tools exist that I can use to explore the data?
27. What are cool ways that I can use the data?
28. Why does the same location occur multiple times?
29. I think I may need an access key to access data stored in S3. Do I?
31. Where can I find people in the OpenAQ Community to chat with?
32. How do I join your Slack Channel?
33. I have a question that isn't answered here. How can I get it answered?
35. What are your favorite emoji?
36. How can I take the Community Survey?
37. How can I find the results from last year's Community Survey?
0. What is OpenAQ?
OpenAQ is the world's first open, real-time and historical air quality platform, aggregating government-measured and research-grade data, as well as low-cost sensor data - entirely open-source. We also help the worldwide air quality community work with these data to fight air inequality - the unequal access to clean air. Our organization strives to be a cheerleader and connector for organizations and individuals around the world working in their communities to fight air pollution. So far, we've done this both through cultivating a virtual community, as well as by holding workshops in places like Delhi, Ulaanbaatar, and Sarajevo. We've also come together as a community to speak out on the role governments and researchers can play in opening up air quality data, and how we, as a community, can help.
OpenAQ envisions a world where everyond breathes healthy air, enabled by universal access to data. Through open data, open-source tools and cooperation, we enable the science, communications and policy necssary to fight air pollution.
1. Why does OpenAQ exist?
Air inequality – unequal access to clean air – is responsible for one out of every 8 deaths in the world. The injustice of this air pollution is often compounded by a lack of easy access to data in the most polluted places – access that, from Los Angeles to Bangkok, has historically enabled communities to improve their air. Many governments publicly share air quality data but often in disparate or temporary forms. The OpenAQ community began capturing these data in 2015 and put them in a universal format for anyone to access through our open-source platform creating the only dataset of its kind open to the world. Years later, ours is the only air quality data platform that meets all four of the following criteria: aggregating disparate data sources at the station level, making data freely available programmatically on a fine temporal level, keeping underlying data-fetching software fully open, and sharing data in physical units rather than as an air quality index. This makes us the go-to data air quality platform for many.
Using data, open-source tools, online engagement and workshops, OpenAQ has convened various sectors to fight air inequality. We share resources and ideas developed locally at workshops and through other convenings with our global community.
We do this because we think this platform can be transformational for public health and the environment.
2. Who started OpenAQ and what motivated them to do it?
The seed of the idea for OpenAQ emerged from a small open air quality project in Ulaanbaatar, Mongolia, launched by co-founders Joe Flasher and Christa Hasenkopf along with Mongolian colleagues. Amazed at the outsized-impact a little open air quality data can have on a community, Christa, an atmospheric scientist, fell in love with open air quality projects. A few years later, she and Joe, a software developer, wondered: what would happen if all of the world’s air quality data were made available for the public to explore? One day, they quit wondering, started building, and began asking passionate people around the world to help.
3. Who is part of OpenAQ and what have they built already?
You can read more about the OpenAQ team here.
Anyone who is using data aggregated from the OpenAQ system, helping build the platform, or building on top of the platform in some fashion, and/or interacting with our community around air inequality is part of the OpenAQ Community. Here's a map of our platform's usage to date (accessed Nov 2019) from Google Analytics:
To get a sense of who is involved with the OpenAQ Platform and using it:
- Visit our repos in GitHub.
- Come chat with our community on Slack.
- See our Community Page that highlights some of the activities and resources created by our community.
4. What type of organization is OpenAQ and where is it located?
OpenAQ was fouonded in Washington, DC. As of 2018 we have held a 501(c)3 nonprofit status. Our EIN is 47-5324172.
The official address of OpenAQ is: 4301 50th St NW, Suite 300 #1046, Washington, DC 20016. Our team works remotely.
5. How is OpenAQ funded?
OpenAQ relies on a combination of government, private-sector, and non-profit/foundation grants in the open data, air quality and general environmental and public health spaces.
We seek to partner with organizations that are interested in fostering air quality awareness and actions around the world. We also expect large organizations that use the OpenAQ platform to find a way to give back to our community for the sustainability of our system. If your organization is interested in sponsoring our work or collaborating in some way, please let us know (info@openaq.org).
We also accept individual donations. We are always on the lookout for partnerships (of all types) to help us sustain and expand what our community is doing.
6. Why does OpenAQ require funding, anyway?
OpenAQ needs funds to a) fetch, store, and serve out all these data, b) keep the system in tip-top shape and develop cool enhancements that make the data easier to use, c) convene air quality enthusiasts around the world and lead an annual Community Ambassador program that trains up-and-coming air quality leaders, d) pay our four full-time staff members, and e) cover a small bit of organizational-upkeep costs.
7. Can I donate funds?
It's awesome you'd even think of doing this! We are honored that you'd ask. Currently, there are two ways that you can help us.
-
Contribute your time and expertise: As a community-driven platform, your contributions and feedback have a huge impact on the long-term sustainability of the OpenAQ infrastructure. In the spirit of supporting an open source culture, here are a few ways that you can get involved:
- Contribute to the OpenAQ platform by adding data sources, fixing bugs, among other valuable data infrastructure support. Learn how to get started as a contributor.
- Share feedback on the tools and other platform features we build with info@openaq.org.
- Help spread the word about OpenAQ to your community, your government or other like-minded organizations that are interested in fostering air quality awareness and actions around the world.
-
Make a financial donation to OpenAQ: As an open-source organization at the forefront of fostering data transparency, we don’t extract money from our database users. We do rely on a combination of institutional funders and valuable contributions from our community. If you are a funding organization or company interested in supporting our work, please reach out to us at info@openaq.org.
We also accept individual donations. Learn more about how to give here. Individual financial donations will lead to greater awareness of work and impact, and provide the type of flexible, unrestricted funding needed for us to continue to grow, innovate and better serve the OpenAQ Community, while remaining a free, open-source resource for everyone.
8. Do you have a Community Code of Conduct?
Why, yes we do! Read the OpenAQ Community Code of Conduct here.
9. What kinds of data are stored to the OpenAQ Platform? Where is Air Quality Index data?
Our focus is air quality data and we aggregate primarily PM2.5, PM10, CO, SO2, NO2, O3, and BC measurements from reference-grade monitors and low-cost sensors. For a limited set of locations, we also have data for PM1, PM1 counts, PM2.5 counts, PM10 counts, CH4, CO mass, CO2, NO, NOx, NO2 mass, O3 mass, SO2 mass, UFP count. Read more about our data format and data type definitions. You can read more about the sources of these data here. We also do not store AQI data, you can find more about why here.
10. What are low-cost sensors and why have they been added to the platform?
In a few words, low-cost sensors are newer, often smaller and less expensive air quality sensors than traditional reference-grade monitors. Low-cost sensors measure specific air pollutants, typically particulate matter and occasionally gaseous pollutants. While "low-cost" is subjective, the cost tends to be $100s or $1000s instead of $10,000s.
In greater detail, low-cost sensor data refer to all non-reference-grade data that is produced by a wide variety of sources, from individuals in their communities, to researchers, to government actors. Low-cost sensor data does not refer directly to specific cost bands, as what counts as “low-cost” varies based on socioeconomic and regional variables. Sensors in this category use technology components that are typically less expensive to manufacture than reference sensors or use reference-grade components in an atypical configuration and, as a consequence, may generate less robust data with lower accuracy and precision.
Learn more through the EPA’s Air Sensor Toolbox and South Coast AQMD’s AQ-SPEC.
Initially, OpenAQ's focus was exclusively reference-grade government air quality data, but we recognized the limitations this poses and how low-cost sensors have become an important part of the air quality ecosystem:
-
Currently, only half of the world’s governments produce air quality data, leaving 1.4 billion people without access to government-sponsored, reference grade data. Low-cost sensors can help fill in these large data gaps, particularly in the Global South, where air quality data has had historically sparse coverage.
-
There are a growing number and variety of stakeholders who are generating air quality data. Advancements in sensor technology and subsequent emergence of lower-cost sensors have lowered the barrier for entry, encouraging a greater number of participants to produce air quality knowledge (including researchers, community organizers, citizen scientists). We wanted the platform to reflect the diversity of the landscape, highlighting new collaborations and possibilities.
Our low-cost sensor platform is a pilot! Give us feedback and let us know how you are using the data.
11. How are data attributed to their sources? How can I find out the sources of the aggregated data?
Data are attributed directly to their sources in the data format. This is evident when downloading data by CSV or accessible from the API.
You can get metadata about a source by using the location endpoint of the API. Updates to metadata can be made here.
If you notice any inconsistencies or mistakes regarding attribution, please let us know at: info@openaq.org
12. How are the data licensed?
Air quality data is factual in nature, and in some jurisdictions may not be subject to copyright or other protections, limiting its use or distribution. However, in some jurisdictions, copyright and/or laws and regulations may apply to some of the data on the OpenAQ platform.
A number of our sources provide their air quality data under specific licensing, such as Creative Commons licensing or open government licenses, which require source attribution. Regardless of such requirements, we believe in the importance of attributing data sources and strongly encourage everyone to do so. Read our Data Policy for more information.
13. How long are data stored?
Our intention is to permanently store and provide these data.
14. What guarantees of the data accuracy are made of the data?
We don't make any guarantees for data quality; that is not the goal of this platform. The goal of this platform is to create a base layer aggregating existing reported government-level, research-grade and other air quality data sources in the same universal format.
A few more details on why we don't do any QA/QC'ing of data: (1) Adjusting these values, even for some form of QA/QC, makes a judgement on a given source's (e.g., government's) data. We do not seek to do that. (2) There are many ways one can do QA/QC'ing of the data, depending on one's need and field, and (3) There are uses for the 'uncleaned' data, one of which is full transparency to downstream users.
We do have a Community Wishlist with ideas of what people would like to see built around the platform and a QA/QC'd layer is on that. If you're interested in seeing this exist, we'd love to help you build it!
15. What are the ways I can access data?
- Use our open API: This can return data from the past two years.
- Download data by csv: This can return data from the past two years.
- You can build you own air quality comparisons of stations around the world.
- Check out the global map
- Realtime JSON dumps
- Daily PostgreSQL database snapshots on Amazon RDS
- Realtime notifications
- Whole-archive query tool
16. Why can I only access data for the last 2 years? Where's the rest of the data?
Second question first! Don't worry, the full archives data are still stored and available through the following mechanisms:
- Go here for access to the data as it's added to our system: https://openaq-fetches.s3.amazonaws.com/index.html. This data is updated every 10 minutes when a new fetch runs. This source also contains all the information for each measurement.
There are very powerful tools to query the above data. One example is using a distributed query tool like Athena or Spark. An example of how to do that can be found here.
So why can you only access the last two years of data through the API and download portal on the OpenAQ website?
It comes down to keeping costs down and maintaining API responsiveness. As the number of our data records grew, it posed a challenge to make all of that information available through an efficiently running API (which also powers the download portal on the site). It also costs us a chunk of change to support a big database, so like a lot of open data projects, we opted to store older data in a more cost-effective method that lets our API run more reliably and our database be smaller.
17. How do I cite the OpenAQ platform in peer-reviewed literature?
Thanks for asking! Once your work is published, please let us know - especially if it is open access. Your work is an important example that justifies our very existence! Also, less grandiosly, it's just cool to hear about and share with the community.
Oh yeah, how to cite the platform. Here are some suggestions:
-
For a single originating data source: "These data were accessed from the OpenAQ Platform (openaq.org) and originate from [XXXX source] (e.g. The U.S. Embassy in Kampala)."
-
For multiple originating data sources that are logistically difficult to list: "These data were accessed from the OpenAQ Platform and originate from multiple sources; see openaq.org for list of sources.
Essentially, where possible, please recognize the originating sources of the data, in addition to their access point.
- Another example by a Community member who cited OpenAQ in their own paper:
"OpenAQ, OpenAQ.org [online] Available from: https://openaq.org (Accessed 3 August 2017), 2017."
Check out some of the publications who have cited us for more inspiration!
18. Why is the OpenAQ Platform open-source?
We want people to be able to add to what we’re doing, to improve upon it, and to build on top of it. We also want people to be able to take snippets of our code that are most useful to them for whatever purpose. That’s best accomplished by keeping our source code open. Read more about our guiding principles here.
19. Where can I find the open-source software that utilizes OpenAQ-aggregated data?
Data Browsers & Visualizations:
- OpenAQ Browser: This browser allows the user to sift, sort and visualize OpenAQ-aggregated data in a friendly way, built by Dolugen Buuralda in Ulaanbaatar, Mongolia
API Libraries
- Ropenaq: an R wrapper for the OpenAQ API, built by Maëlle Salmon in Barcelona, Spain
- py-openaq: a Python wrapper for the OpenAQ API, built by David Hagan in Boston, USA
20. What is the license of the software?
All parts of the OpenAQ software are under the MIT License. Each of our GitHub repos lists this under its license. See here for the API, for example.
21. What are some ongoing open-source projects involving OpenAQ?
Check out our community page here
Do you have an open-source tool you'd like to share? Let us know! Make an issue in GitHub or just email info@openaq.org
22. How can I help contribute new data sources? What information is needed for a new data source?
Thanks for being interested in sharing a data source! You can make a GitHub issue here, submit something via this Google Form, or email us at info@open.org. The data format and requirements are listed here.
Are you part of a government entity seeking to share data? First of all THANKS! Read this post with step by step instructions.
23. I suggested a new data source, but I haven't seen it added yet. What gives?
For government-grade sources, we generally have more data sources waiting to be added than we do bandwidth to insert them. We're always seeking help from those interested and with the skills to add in data sources to our system. More info on how to get started as a contributor can be found here.
For research-grade data, we're working on an automatic way to add in data, but otherwise it takes some time to add it in by hand.
For non-research-grade low-cost sensors, we are evaluating the best way to add in the data responsibly (it's tricky business) and are planning a pilot soon.
24. What skills are needed to contribute to the code-base?
The front end is written in CSS, HTML and JavaScript using React. The API and updater are written in JavaScript using Node.js. There are a number of other small pieces (CSV exporter, health checker), mostly written in JS/Node.js. If you're interested in system architecture, everything runs on AWS.
25. How can I contribute my software development skills? Where can I find existing GitHub issues I can help out with?
Here are Contribution Guidelines for developers.
Here's where we are on GitHub. You can see the most important respositories pinned at the top and all have managed issues lists. And one of the best ways to get oriented right now is to just reach out on Slack.
26. What community tools exist that I can use to explore the data?
You can start playing around by building your own air quality comparisons of stations around the world, check out the global map or downloading data by csv or our open API.
Here are open-source tools built by community members: Data Browsers & Visualizations:
- OpenAQ Browser: This browser allows the user to sift, sort and visualize OpenAQ-aggregated data in a friendly way, built by Dolugen Buuralda in Ulaanbaatar, Mongolia
API Libraries
- Ropenaq: an R wrapper for the OpenAQ API, built by Maëlle Salmon in Barcelona, Spain
- py-openaq: a Python wrapper for the OpenAQ API, built by David Hagan in Boston, USA
Do you have an open-source tool you'd like to share? Let us know! Make in issue in GitHub or just email info@openaq.org
27. What are cool ways that I can use the data?
That's really up to you! And your imagination is the limit. In addition to the tools in the above question, you can find out how our community is using the data on our Community Page, our blog and just by chatting with people on Slack.
You can start playing around by building you own air quality comparisons of stations around the world, check out the global map, downloading data by csv or use our open API.
28. Why does the same location occur multiple times?
This normally happens because the location names are slightly different coming from the source. Let's take a look at 'Shastri nagar, Jaipur - RSPCB' as an example.
You can see here that there are slight differences in spacing, which makes our system think these are different locations. Without a universal ID for all instruments, it's difficult to know exactly that these locations are the same. However, you can look at their coordinates (if provided) and make a guess that they're the same location.
29. I think I may need an access key to access data stored in S3. Do I?
The data is in a publicly accessible bucket, so there is no key for us to provide. This sounds like you are trying to use the AWS CLI or an SDK to access the bucket but have not provided your personal AWS account credentials (that you create yourself). If you have an AWS account, you can provide those credentials. But also, you can just use the --no-sign-request flag in the CLI (or similar in SDK) to tell the request to not pass any credentials. That should also allow you to proceed.
30. Do you have dataset descriptions for the datasets listed here - https://openaq-fetches.s3.amazonaws.com/index.html?
The Daily file was a daily rollup of the data coming into the system but has been deprecated for some time so you’ll only find older data there. You likely want to use the realtime or realtime-gzipped prefix (same data, just one is compressed and the other is not). This data comes in ~10 minute intervals as it is inserted into our system. There’s a folder with all the files for every day, all of the files are named with a timestamp for when the file was created. So if you’re searching for specific days or looking to do aggregations, the best way to do so would be to use AWS Athena, this guide https://medium.com/@openaq/how-in-the-world-do-you-access-air-quality-data-older-than-90-days-on-the-openaq-platform-8562df519ecd may be helpful.
Broadly, the data in those files conforms to the format laid out in https://github.com/openaq/openaq-data-format/.
31. Where can I find people in the OpenAQ Community to chat with?
The most substantive way to interact with the community is to join our Slack Channel. If you're not familiar with Slack, it's just a free tool we use to chat with one another.
You can also join us on Twitter or on our Facebook Page.
Always feel free to reach out by email, too (info@openaq.org).
32. How do I join your Slack Channel?
Go here: https://join.slack.com/t/openaq/shared_invite/zt-gq14aew7-CVdp131g7TR7o9iiXIVDLwGeneral/
33. I have a question that isn't answered here. How can I get it answered?
The fastest way is to join our Slack channel and ask us and the rest of the community. The second fastest way is to email info@openaq.org.
Slack is faster than email because you'll get to talk to the whole community who often know more than any single one of us and also are in various timezones. At this point, Community members are spread over many parts of the world.
34. I've made something awesome with data aggregated onto the OpenAQ Platform. Do you want to know about it?
Is the sky blue (on an unpolluted day)? YES!! We are always interested to hear how people are using these aggregated data - in fact, conveying the awesome work going on in our community is how we justify our existence to both ourselves and our funders! So please do tell us (info@openaq.org).
35. What are your favorite emoji?
That's such an odd and awesome question. Here you go: 👍💃🐶🚀