Lambda canterbury #946
Merged
Lambda canterbury #946
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
resolves #756 |
|
fixed bugs |
russbiggs
requested changes
Feb 15, 2023
src/sources/nz.json
Outdated
| { | ||
| "url": "http://data.ecan.govt.nz/data/29/Air/Air%20quality%20data%20for%20a%20monitored%20site%20(10%20Minute)/JSON?SiteID=$station&StartDate=$date&EndDate=$date", | ||
| "adapter": "canterbury", | ||
| "name": "New Zealand Canterbury", |
There was a problem hiding this comment.
use a more descriptive name for this. What is the name of the data provider instead of where it covers
|
@majesticio lets do a wiki entry for this source before merging |
|
changes have been made, wiki is up https://github.com/openaq/openaq-fetch/wiki/Canterbury-NZ-Environment-Canterbury |
|
let revert name to a more tag-like name maybe "canterbury-nz" @majesticio |
russbiggs
added a commit
that referenced
this pull request
Apr 10, 2023
* Updated all important packages to latest versions 2 packages were not updated because they are only used in adapters that at no longer active and do not work. I left them for now but we could probably remove them at some point. * Moved files to src folder for easier deployment Instead of having to exclude a bunch of files in the deployment code * Added adapter-update.md to ignore file * Updated to support being triggered from SQS * Added method to update the database provider data * Removed migration data Because these were from the old database and the new database has some of the same table names but does not include the migration table I wanted to make sure that these would not cause issues now that we are using knexjs to update the provider table when we change the source config * Updated all active adapters to es6 import/export syntax * Updated turkey adapter * Added a response check to caaqm Meant to throw proper error in the case where no data was returned * Added more source options * Added some more functionality More error checking, ability to pass time * Added the offset option to fetch method * Updated the null country to be 99 from -- * Cleaned up adapater to throw proper errors refer to issue #918 for more details * Updated tuzlanski logging methods Added more error messages to make it easier to debug. See issue #919 * Removed the adapater error from au_sa Because of the try/catch block it means that non fatal errors become fatal * Added new ae adapter and modified to work with lambda method * Added turkiye adapter and updated to work with lambda method * Updated the deployment method to support multiple deployments * Updated kosovo to work with lambda but then turned off * Increased visibility timeout Must be equal to or greater than the lambda timeout * corrected attribution, location site name * Israel adapter going live * Revert "Israel adapter going live" This reverts commit 1cc3756. * kosovo fetch incomplete * kosovo adapter * works * working adapter * removed old kosovo adapter * added data source comment * added source.url. adapter should be ready for PR * padded hours for luxon date format * fixed adapter and changed source 'active'='true' * bug fix - units for carbon monoxide to µg/m³ (#928) * Moved the certs directory * Adding datetime/offset to eea-direct * Added the ability to limit fetch to an adapter Adapter is somewhat analagous to provider. This lets us deploy a fetcher for a set of sources that are tied to one adapter. * Added missing part to pass adapter via env file * Updated the schedular to use the deployments method Deployment will be a list of fetching jobs that we need to send an SQS message out for. * Commiting missing files * New Cyprus lambda adapter (#953) * Cyprus ready for review / merge. license-> cy.json * simplified luxon by using DateTime.fromFormat * requested changes for PR, update to adapter name and attribution --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * repo cleanup * Modified the script to update providers and adapters * Hungary adapter (#958) * hungary adapter --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Lambda canterbury (#946) * canterbury * fetcher for Canterbury, NZ ready for review, merge * ready for review/deploy * updated moment to luxon, cleaned up * made requested changes for PR * update to adapter name --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Lambda trinidad tobago (#949) * first commit.needs review credit magsyg * credit @magsys PR #754 * IP updated stations and added parameters. * C - adapter works ready for PR * added params names to ID's * update * updated moment to luxon * updated adapter name and attribution * updated adapter name * updated Number to parseFloat --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Bug/south australia (#957) * Draft: almost ready * updates to attribution and units * ready for review/PR * added modules for rss and html parsing * update to averaging period 'co' = 8, adapter name --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Lambda Serbia fetcher (#951) * first commit, works-need review for dupes w/eea * complete adapter ready for review/deploy * updated moment to luxon * updated Number to parseFloat * updated adapter name --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Lambda catalonia adapter (#950) * added catalonia, needs review * will close #711 and #709 after PR. review 4 dupes * resolved requested changes * updated adapter name * added params 'no', 'pm1' * updated moment to luxun, added 'no, nox, pm1' enum --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Israel new fetcher lambda (#959) * israel envista-new * added parameters pm10 and pm25 * commented out print * optional params * almost ready * ready for review/PR * removed moment * requested changes for PR --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * added export const name = 'kosovo' (#961) Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Removed the AWS variables * Bug/date Hungary (#962) * corrected discrepancy with utc/local time * cleaned up --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * real date fix final (#963) Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * added function to remove duplicate stations (#966) * added function to remove duplicate stations * requested changes * cleaned up filterDuplicates() --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * removed duplicate locations less than 0.1 km (#967) * removed duplicate locations less than 0.1 km * requested changes * cleaned up filterDuplicates() --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * cdk updates * update to averaging period (#968) * update to averaging period * removed comment --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * Rio adapter! (#964) * Rio adapter! * cleaned up --------- Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com> * cdk fixes and deployment updates * depcheck gh action * added to ci.yml * yml update * update * update ci.yml * added || true * output to json * formatting updates, commonjs removal * added script for ci workflow * fix ci workflow * debug * update to script * update * update to script * update to workflow * debug * debug print * update workflow * commonjs to modules * Delete .DS_Store * decode 64 * update * update to workflow * hail mary * quotes unneeded * echo -e for escape * printf "%b" escape * escape * hail mary * pipe to echo * printf string formatting * update * back to square one * yaml * need to figure out escape character env.PR_COMMENT * no pipe * not escaping new line * Update process_depcheck_output.sh * Update process_depcheck_output.sh * Update process_depcheck_output.sh * Update process_depcheck_output.sh --------- Co-authored-by: Christian Parker <cparker@talloaks-environmental.com> Co-authored-by: Gabe Fosse <gabofosse55@gmail.com> Co-authored-by: Gabriel Fosse <67290377+majesticio@users.noreply.github.com> Co-authored-by: Gabriel Fosse <kraken+majesticio@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #650
credit to @magsyg
based off of #756