Add secure Geoserver access #242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Thanks for submitting a PR.
Make sure you have looked at CONTRIBUTING guidelines.
dbyrns
approved these changes
Apr 19, 2022
tlvu
requested changes
Apr 20, 2022
There was a problem hiding this comment.
I would like to ask for a favor, could you make this change an optional component (under https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components dir)?
The reason is I already planned to eventually break out every components into separate components to allow for each org to enable only the parts they need so I'd rather have any new changes aligned with the future plan.
You can look into https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse/optional-components/generic_bird for sample how it's done. It has hooks into the front Nginx proxy and Magpie as well so it's a pretty complete example to based upon to save you time.
Document your new optional component the README.rst at the root of the optional-components/ dir.
Also, do not forget to prepare the https://github.com/bird-house/birdhouse-deploy/blob/master/CHANGES.md file so the PR is complete for eventual release.
The full release procedure is documented here https://github.com/bird-house/birdhouse-deploy/tree/master/birdhouse#release-procedure
Sorry for the last minute refactoring request. DavidB has briefly mentioned this upcoming feature during our previous bi-weekly but I forgot to inquire about the implementation whether it will be a "component".
Will be implementing as optional component
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/885/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/886/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/887/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/888/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/889/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
tlvu
approved these changes
Aug 24, 2022
There was a problem hiding this comment.
Oh sorry I did not see you had done the optional-component.
Everything looks good, one minor thing missing is updating the doc of the optional-components https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/optional-components/README.rst to list and describe your component.
See many example there for the format.
Should do this after merging with master. I think FrancisM has added a new optional-component lately as well.
|
@tlvu Just to be sure, since I don't do merges in this repo often, all that's left is to run bumpversion before merging the pull request? |
https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/README.rst#release-procedure (added recently after you opened this PR so you might not be aware). You are basically at this last step "Only when you are ready to merge the PR immediately, ...". Follow from there. If something unclear, let me know. |
|
@francisPLT |
Yes, I just saw that, working on it |
github-actions
bot
added
ci/operations
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/996/Result : success BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-216.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/808/NOTEBOOK TEST RESULTS |
tlvu
reviewed
Aug 24, 2022
| @@ -242,7 +242,7 @@ PLATFORMS = { | |||
| 'info': { | |||
| 'name': 'PAVICS', | |||
| 'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', | |||
| 'version': '1.21.1', | |||
| 'version': '1.22.0', | |||
| 'releaseTime': '2022-08-24T15:43:51Z', | |||
There was a problem hiding this comment.
@fmigneault release time did not update. Is there a special option we need to run bumpversion with, for release time to update? If yes, we need to doc this special option. I just checked one of my previous release, the release time did not update for me as well. Or need a special version of bumpversion?
There was a problem hiding this comment.
Seems like the portion https://github.com/bird-house/birdhouse-deploy/blob/master/.bumpversion.cfg#L28-L29 cannot update itself because bumpversion rewrites the .bumpversion.cfg file after applying changes using what it initially read. So the update works the first time when datetime match, but fail on following cases because the new datetime was not applied.
I think we could hack around this using a separate file that contains only the release time. Bumpversion could modify that file without problem. It would need to be mounted in the docker image for the python config to read it.
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/997/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
|
@francisPLT FYI, the tag should be on the merge commit created by merging this PR, not on the bump version commit. It's okay, just for next time. |
@francisPLT Just to be clear, do not touch the tag at the wrong location anymore (FrancisM said to remove the tag at the wrong location and redo it but I've merged this to our production fork already and sync the tags, so too late to fix the wrong location of the tag). |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1000/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1001/Result : failure BIRDHOUSE_DEPLOY_BRANCH : add-secure-geoserver-access DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
|
@francisPLT Will need to apply this new component to |
3 tasks
fmigneault
added a commit
that referenced
this pull request
Sep 1, 2022
## Overview Because `bumpversion` cannot update its own `.bumpversion.cfg` file to apply the new `releaseTime` value (since it rewrites the config with its own config handler once changes were applied), use a workaround to produce the same effect. Use Makefile to parse the current release time, then run `bumpversion` as normal, retrieve the new release time that was applied by `bumpversion`, then replace the old->new release time in the config for future calls. To test, a few bumps were done (note: second tag on the line is the "active version" on that commit) ``` * eb537ef Bump version: 1.22.3 → 1.22.4 * f41fb55 Bump version: 1.22.2 → 1.22.3 * 81470bd Bump version: 1.22.1 → 1.22.2 ``` The diffs are as follows, showing correct behaviour to replace new `releaseTime` everywhere: <details><summary> diff 1.22.2 → 1.22.3 </summary> <p> ```diff diff --git a/.bumpversion.cfg b/.bumpversion.cfg index 36e736a..8a68e80 100644 --- a/.bumpversion.cfg +++ b/.bumpversion.cfg @@ -1,23 +1,23 @@ [bumpversion] -current_version = 1.22.2 +current_version = 1.22.3 commit = True tag = False tag_name = {new_version} parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+) (?P<releaseTime>.*)? -serialize = +serialize = {major}.{minor}.{patch} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:file:CHANGES.md] -search = +search = [Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest) ------------------------------------------------------------------------------------------------------------------ -replace = +replace = [Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest) ------------------------------------------------------------------------------------------------------------------ - + [//]: # (list changes here, using '-' for each new entry, remove this when items are added) - + [{new_version}](https://github.com/bird-house/birdhouse-deploy/tree/{new_version}) ({utcnow:%Y-%m-%d}) ------------------------------------------------------------------------------------------------------------------ @@ -30,11 +30,11 @@ search = {current_version} replace = {new_version} [bumpversion:file:RELEASE.txt] -search = {current_version} 2022-08-25T03:24:48Z +search = {current_version} 2022-08-25T03:27:35Z replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:part:releaseTime] -values = 2022-08-25T03:24:48Z +values = 2022-08-25T03:27:35Z [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template] search = 'version': '{current_version}' diff --git a/CHANGES.md b/CHANGES.md index 5a0f6f9..357ccfc 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -16,6 +16,11 @@ [//]: # (list changes here, using '-' for each new entry, remove this when items are added) +[1.22.3](https://github.com/bird-house/birdhouse-deploy/tree/1.22.3) (2022-08-25) +------------------------------------------------------------------------------------------------------------------ + +[//]: # (list changes here, using '-' for each new entry, remove this when items are added) + [1.22.2](https://github.com/bird-house/birdhouse-deploy/tree/1.22.2) (2022-08-25) ------------------------------------------------------------------------------------------------------------------ diff --git a/Makefile b/Makefile index dbe3146..35367e9 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # Generic variables override SHELL := bash override APP_NAME := birdhouse-deploy -override APP_VERSION := 1.22.2 +override APP_VERSION := 1.22.3 # utility to remove comments after value of an option variable override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g") diff --git a/README.rst b/README.rst index 8b79614..533d425 100644 --- a/README.rst +++ b/README.rst @@ -14,13 +14,13 @@ for a full-fledged production platform. * - releases - | |latest-version| |commits-since| -.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.2.svg +.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.3.svg :alt: Commits since latest release - :target: 1.22.2...master + :target: 1.22.3...master -.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.2-blue.svg?style=flat +.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.3-blue.svg?style=flat :alt: Latest Tag - :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.2 + :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.3 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest :alt: ReadTheDocs Build Status (latest version) diff --git a/RELEASE.txt b/RELEASE.txt index bc149ee..c7dc42e 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -1 +1 @@ -1.22.2 2022-08-25T03:24:48Z +1.22.3 2022-08-25T03:27:35Z diff --git a/birdhouse/config/canarie-api/docker_configuration.py.template b/birdhouse/config/canarie-api/docker_configuration.py.template index 111582f..7b37a80 100644 --- a/birdhouse/config/canarie-api/docker_configuration.py.template +++ b/birdhouse/config/canarie-api/docker_configuration.py.template @@ -17,8 +17,8 @@ SERVICES = { 'info': { 'name': 'Node', 'synopsis': 'Nodes are data, compute and index endpoints accessed through the PAVICS platform or external clients. The Node service is the backend that allows: data storage, harvesting, indexation and discovery of local and federated data; authentication and authorization; server registration and management. Node service is therefore composed of several other services.', - 'version': '1.22.2', - 'releaseTime': '2022-08-25T03:24:48Z', + 'version': '1.22.3', + 'releaseTime': '2022-08-25T03:27:35Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', @@ -242,8 +242,8 @@ PLATFORMS = { 'info': { 'name': 'PAVICS', 'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', - 'version': '1.22.2', - 'releaseTime': '2022-08-25T03:24:48Z', + 'version': '1.22.3', + 'releaseTime': '2022-08-25T03:27:35Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', ``` </p> </details> <details><summary> diff 1.22.3 → 1.22.4 </summary> <p> ```diff diff --git a/.bumpversion.cfg b/.bumpversion.cfg index 8a68e80..f0638ae 100644 --- a/.bumpversion.cfg +++ b/.bumpversion.cfg @@ -1,5 +1,5 @@ [bumpversion] -current_version = 1.22.3 +current_version = 1.22.4 commit = True tag = False tag_name = {new_version} @@ -30,11 +30,11 @@ search = {current_version} replace = {new_version} [bumpversion:file:RELEASE.txt] -search = {current_version} 2022-08-25T03:27:35Z +search = {current_version} 2022-08-25T03:27:50Z replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ} [bumpversion:part:releaseTime] -values = 2022-08-25T03:27:35Z +values = 2022-08-25T03:27:50Z [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template] search = 'version': '{current_version}' diff --git a/CHANGES.md b/CHANGES.md index 357ccfc..467ee61 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -16,6 +16,11 @@ [//]: # (list changes here, using '-' for each new entry, remove this when items are added) +[1.22.4](https://github.com/bird-house/birdhouse-deploy/tree/1.22.4) (2022-08-25) +------------------------------------------------------------------------------------------------------------------ + +[//]: # (list changes here, using '-' for each new entry, remove this when items are added) + [1.22.3](https://github.com/bird-house/birdhouse-deploy/tree/1.22.3) (2022-08-25) ------------------------------------------------------------------------------------------------------------------ diff --git a/Makefile b/Makefile index 35367e9..c374b08 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # Generic variables override SHELL := bash override APP_NAME := birdhouse-deploy -override APP_VERSION := 1.22.3 +override APP_VERSION := 1.22.4 # utility to remove comments after value of an option variable override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g") diff --git a/README.rst b/README.rst index 533d425..e74149c 100644 --- a/README.rst +++ b/README.rst @@ -14,13 +14,13 @@ for a full-fledged production platform. * - releases - | |latest-version| |commits-since| -.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.3.svg +.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.22.4.svg :alt: Commits since latest release - :target: 1.22.3...master + :target: 1.22.4...master -.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.3-blue.svg?style=flat +.. |latest-version| image:: https://img.shields.io/badge/tag-1.22.4-blue.svg?style=flat :alt: Latest Tag - :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.3 + :target: https://github.com/bird-house/birdhouse-deploy/tree/1.22.4 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest :alt: ReadTheDocs Build Status (latest version) diff --git a/RELEASE.txt b/RELEASE.txt index c7dc42e..902af90 100644 --- a/RELEASE.txt +++ b/RELEASE.txt @@ -1 +1 @@ -1.22.3 2022-08-25T03:27:35Z +1.22.4 2022-08-25T03:27:50Z diff --git a/birdhouse/config/canarie-api/docker_configuration.py.template b/birdhouse/config/canarie-api/docker_configuration.py.template index 7b37a80..536a706 100644 --- a/birdhouse/config/canarie-api/docker_configuration.py.template +++ b/birdhouse/config/canarie-api/docker_configuration.py.template @@ -17,8 +17,8 @@ SERVICES = { 'info': { 'name': 'Node', 'synopsis': 'Nodes are data, compute and index endpoints accessed through the PAVICS platform or external clients. The Node service is the backend that allows: data storage, harvesting, indexation and discovery of local and federated data; authentication and authorization; server registration and management. Node service is therefore composed of several other services.', - 'version': '1.22.3', - 'releaseTime': '2022-08-25T03:27:35Z', + 'version': '1.22.4', + 'releaseTime': '2022-08-25T03:27:50Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', @@ -242,8 +242,8 @@ PLATFORMS = { 'info': { 'name': 'PAVICS', 'synopsis': 'The PAVICS (Power Analytics for Visualization of Climate Science) platform is a collection of climate analysis services served through Open Geospatial Consortium (OGC) protocols. These services include data access, processing and visualization. Both data and algorithms can be accessed either programmatically, through OGC-compliant clients such as QGIS or ArcGIS, or a custom web interface.', - 'version': '1.22.3', - 'releaseTime': '2022-08-25T03:27:35Z', + 'version': '1.22.4', + 'releaseTime': '2022-08-25T03:27:50Z', 'institution': 'Ouranos', 'researchSubject': 'Climatology', 'supportEmail': '${SUPPORT_EMAIL}', ``` </p> </details> ## Changes **Non-breaking changes** - Adds Makefile to run basic DevOps maintenance commands on the repository - Adds RELEASE.txt with the active release tag and datetime - Replace `now:` directives by `utcnow:` to report time properly according to employed ISO format **Breaking changes** - n/a ## Related Issue / Discussion - Resolves problem described here: #242 (comment)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
I added a new provider and location as an optional component,
test-geoserver-secured-access, in order to test secured access to Geoserver before we completely move Geoserver behind Twitcher (undetermined date).The old
/geoserverpath is still available, so current workflows should not be affected.The new
/geoserver-securedpath is now available for testing once the optional component is activated.To test the
geoserver-securedservice through Magpie, each workspace needs to be added to the new service and then permissions can be set on a per-workspace or even layer basis.Service configuration:
Ex: for the Anonymous Group configuration:
From the above example, a
GetFeaturerequest for a layer in thepublicworkspace will succeed for any user using any of these two request types:Whereas access to the
privateworkspace will require a user or group be given explicit permissions.Mind you, I'm not an expert with Nginx, so I based these changes from the existing configurations.
Changes
Non-breaking changes
geoserver-secured/geoserver-securedBreaking changes
Related Issue / Discussion
Future considerations
{BASE_URL}/geoserver/web). I thought I had found a way a quick way to manage permissions to it, but further testing proved me wrong. All the internal redirections while navigating the web console will probably create issues, and the UI itself did not appreciate the proxy/redirect. I have not looked into this further for the time being.