Refactor bcgov/api and merge to bcgov/wps #432

Kyubinhan · 2020-08-15T00:22:14Z

Move all the API files under api/
Update sonar-project.properties to narrow down the coverage focus
Update Openshift scripts (paths and env variables)
Remove bash scripts and common folder at root
Update fireweather_bot.py to raise more exceptions
Add more commands in Makefile for docker environment
Copy all the files at the last step in dev.Dockerfile

Skeleton for API project

Add missing compliance audit file

Added stations and percentiles (reading from json files)

* Add basic script for calculating 90th percentile on sample data * Revised script & output based on Brady's initial feedback * Add start & end time ranges as parameters * Add Jupyter notebook for script interaction * Modified script to calculate percentiles offline; add results as json file * Clean out unneeded files * Edits made based on Sy's feedback for PR

- All weather station names. - Extracting csv from access database. - Update Pipfile.lock. - Fixed unit tests. - Added sample environment file.

- formatting station json nicely - exclude stations starting with ZZ (inactive) and ending with QD (quick deploy) - ordered result by station name)

* Adding initial pipeline code for deployment to openshift. Updated jenkinsfile and openshift templates. Included ORIGINS env. variable. * Using wps-api for config and deploy scripts. * Updated the match label. * Removed build notifications and using env variable for origins saved in config map. * Updated the definition of data under config map. * Using master branch as reference.

- Re-generated data with year list - Change calculate_percentile_offline.py to comply with pep8 - Added comment on data_years type

…e - WPS #95 (#17) * Generated precalculated percentiles data for 20 year and full range of data available for weather stations * Updated folder names * Updated script to generate precalculated percentile data for multiple year ranges * test commit * Removed default year range * Added new tests to check for blank stations and invalid year range * updated folder name logic and updated run command to run tests before run

* Add updated weather_stations.json after newest script execution * Resolve missed merge with master; add updated data jsons

* Added disclaimer to API description * Deleted empty file

* Cleanup before moving to shared pipeline scripts NOTE: this is not going through the pipeline, because it (temporarily) eliminated the pipeline!

git-subtree-dir: openshift/scripts git-subtree-split: 262ed11 NOTE: There is still no pipeline, since this PR is adding files to get the new pipeline running.

* Copied wps-web Jenkinsfile * Renamed BC and DC for general consumption * BC modified to use standard vars * BC relabeling, cleanup and adjustment * BC and DC revision * Cut down DC and fixed some identation * Updated image path * Removed old Rocket.Chat token * DC allocations, typos and env.origins * Removed stale vars in BC and Jenkinsfile

* Remove uncooperative subtree * Squashed 'openshift/scripts/' content from commit b18e587 git-subtree-dir: openshift/scripts git-subtree-split: b18e587 * Squashed 'openshift/scripts/' changes from b18e587..9a2c8cd 9a2c8cd oc_cleanup.sh typo (#163) git-subtree-dir: openshift/scripts git-subtree-split: 9a2c8cd

and set allowed origins to wps-web-*.pathfinder.gov.bc.ca

* Changed json structure to comply with python conventions. * Changed python code to comply to linting rules. * Added linting rules. * Removed data for stations not used. * Added standardized ways to run test and lint. * Added additional Dockerfile to deal with linting in "dev" build environment. * Added linting commands to Makefile. * Modified some python code to allow linting to pass when developer packages aren't installed. * Updated documentation/README * test.sh and lint.sh working now

* Bump bleach from 3.1.1 to 3.1.2 Bumps [bleach](https://github.com/mozilla/bleach) from 3.1.1 to 3.1.2. - [Release notes](https://github.com/mozilla/bleach/releases) - [Changelog](https://github.com/mozilla/bleach/blob/master/CHANGES) - [Commits](mozilla/bleach@v3.1.1...v3.1.2) Signed-off-by: dependabot[bot] <support@github.com> * Ran pipenv update Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* documented assumptions

script for copying a database from an openshift pod to local

- Renamed models as requested in prior code review (they make a bit more sense now) - Removed all references to "forecast" from the models, in order to reduce confusion between what forecasters forecast, and what weather models predict. - Updated poetry references to latest.

sonarcloud · 2020-08-18T19:14:05Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
6 Code Smells

93.9% Coverage
0.0% Duplication

* Move all the API files under `api/` * Update `sonar-project.properties` to narrow down the coverage focus * Update Openshift scripts (paths and env variables) * Remove bash scripts and common folder at root * Update `fireweather_bot.py` to raise more exceptions * Add more commands in Makefile for docker environment * Copy all the files at the last step in `dev.Dockerfile`

DerekRoberts and others added 30 commits January 29, 2020 18:37

Initial commit

c25e57c

Add compliance audit file

233304c

README template

0fafda4

Skeleton for API (#3)

608168a

Skeleton for API project

Updating compliance audit file

561f01f

Merge pull request #1 from bcgov/repo-mountie/add-compliance

80ed5bd

Add missing compliance audit file

stations & percentiles (#4)

d9fcebd

Added stations and percentiles (reading from json files)

Modified request and response as per request from Kyubin (#5)

67f0836

Generate weather station JSON (#8)

400f64a

- All weather station names. - Extracting csv from access database. - Update Pipfile.lock. - Fixed unit tests. - Added sample environment file.

Exclude inactive and quick deploy stations (#10)

38128dd

- formatting station json nicely - exclude stations starting with ZZ (inactive) and ending with QD (quick deploy) - ordered result by station name)

Allow POST method (#11)

8119f87

README tweaks (#14)

f7cfdcc

Listing years in response (#16)

9520117

- Re-generated data with year list - Change calculate_percentile_offline.py to comply with pep8 - Added comment on data_years type

Add updated weather_stations.json after newest script execution

b47a0ba

Merge branch 'master' of https://github.com/bcgov/wps-api

8cef06e

Merge branch 'master' of https://github.com/bcgov/wps-api

0052a52

Resolve neglected merge (#18)

243daff

* Add updated weather_stations.json after newest script execution * Resolve missed merge with master; add updated data jsons

Merge branch 'master' of https://github.com/bcgov/wps-api

1e7ea77

Added disclaimer to description of API specification WPS #60 (#20)

e7d1f7e

* Added disclaimer to API description * Deleted empty file

Clear out repo before applying shared pipeline scripts (#22)

f80fd07

* Cleanup before moving to shared pipeline scripts NOTE: this is not going through the pipeline, because it (temporarily) eliminated the pipeline!

Squashed 'openshift/scripts/' content from commit 262ed11 (#23)

0e1f9d6

git-subtree-dir: openshift/scripts git-subtree-split: 262ed11 NOTE: There is still no pipeline, since this PR is adding files to get the new pipeline running.

Run oc_deploy script for prod (#28)

b73703d

and set allowed origins to wps-web-*.pathfinder.gov.bc.ca

Merge branch 'master' of https://github.com/bcgov/wps-api

1e23e3d

Sybrand and others added 13 commits August 12, 2020 10:12

Storage assumptions (#131)

1d0441e

* documented assumptions

script for copying a database from pod to local (#133)

8767fa8

script for copying a database from an openshift pod to local

Merge remote-tracking branch 'api/main' into task/merge-wps-api-2/272

35c64a8

Move app into api

4814eaf

Move scripts folder into api

b54b05d

Move alembic folder into api

36f9f28

Move other files

bc580a0

Move README.md to api and revive the one that in root

358fb79

Remove bash scripts in root

081f1a4

Update GH actions

d1f6014

Update Makefile & sonar-project.properties

5cc08f6

Update build.bc.yaml so that it can refer to /api

f55f9c0

Kyubinhan force-pushed the task/merge-wps-api-2/272 branch from a148a9f to f55f9c0 Compare August 15, 2020 00:30

Kyubinhan added 4 commits August 18, 2020 09:37

Remove common folder and update envars

caf5edb

Update fireweather_bot to raise more exceptions

0a13c7e

Update .env.example

11277f3

Address lint errors

a23b376

Kyubinhan changed the title ~~Task/merge wps api 2/272~~ Refactor bcgov/api and merge to bcgov/wps Aug 18, 2020

Copy all files at the last step in dev.Dockerfile

0a81a7b

Kyubinhan force-pushed the task/merge-wps-api-2/272 branch from 665e88a to 0a81a7b Compare August 18, 2020 19:08

Kyubinhan marked this pull request as ready for review August 18, 2020 19:56

Kyubinhan requested review from Sybrand and andrea-williams August 18, 2020 19:56

andrea-williams approved these changes Aug 18, 2020

View reviewed changes

Sybrand approved these changes Aug 18, 2020

View reviewed changes

Kyubinhan merged commit 175e375 into main Aug 18, 2020

Kyubinhan deleted the task/merge-wps-api-2/272 branch August 18, 2020 22:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor bcgov/api and merge to bcgov/wps #432

Refactor bcgov/api and merge to bcgov/wps #432

Kyubinhan commented Aug 15, 2020 •

edited

Loading

sonarcloud bot commented Aug 18, 2020 •

edited

Loading

Refactor bcgov/api and merge to bcgov/wps #432

Refactor bcgov/api and merge to bcgov/wps #432

Conversation

Kyubinhan commented Aug 15, 2020 • edited Loading

sonarcloud bot commented Aug 18, 2020 • edited Loading

Kyubinhan commented Aug 15, 2020 •

edited

Loading

sonarcloud bot commented Aug 18, 2020 •

edited

Loading