Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move ows proxy send_request function under corresponding adapter method #118

Merged
merged 6 commits into from Feb 1, 2023

Conversation

fmigneault
Copy link
Contributor

@fmigneault fmigneault commented Jan 30, 2023

Changes:

  • Add /ows/verify/{service_name}[/{extra_path}] endpoint analoguous to /ows/proxy/{service_name}[/{extra_path}]
    to only verify if access is granted to this service, for that specific resource path, and for the authenticated user,
    without performing the proxied request. This can be employed by servers and external entities to validate that
    authorization will be granted for the user without executing potentially heavy computation or large data transfers
    from the targeted resource that would otherwise be performed by requesting the /ows/proxy equivalent location.
    One usage example of this feature is using NGINX Authentication Based on Subrequest Result to verify an alternate resource prior to proxying a service
    request that needs authenticated access to the first resource.
  • Add the OWS proxy send_request operation under the twitcher.adapter interface to allow it applying relevant
    proxying adjustments when using derived implementation. The DefaultAdapater simply calls the original function
    that was previously called directly instead of using the adapter's method.
  • Removed the extra_path and request_params arguments from OWS proxy send_request to better align them with
    arguments from other adapter methods. These parameters are directly retrieved from the request argument, which was
    also provided as input to send_request.

Purpose

This is used in conjunction of Ouranosinc/Magpie#571
Without this fix, there is no way for Magpie to retrieve the link between the proxied request/response, as this was all contained within send_request.

To Do

Twitcher 0.8.0 will be tagged (and should be released) after merge in order to build the new Docker required by Magpie.

@fmigneault fmigneault self-assigned this Jan 30, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jan 30, 2023

Codecov Report

Merging #118 (80dc1c2) into master (8c2bf25) will decrease coverage by 0.41%.
The diff coverage is 50.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master     #118      +/-   ##
==========================================
- Coverage   74.20%   73.80%   -0.41%     
==========================================
  Files          41       41              
  Lines        1795     1821      +26     
==========================================
+ Hits         1332     1344      +12     
- Misses        463      477      +14     
Impacted Files Coverage Δ
twitcher/owsproxy.py 61.31% <45.00%> (-3.15%) ⬇️
twitcher/adapter/base.py 53.33% <50.00%> (-0.24%) ⬇️
twitcher/utils.py 81.35% <50.00%> (ø)
twitcher/adapter/default.py 75.67% <62.50%> (-3.64%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@fmigneault fmigneault merged commit 9aa9956 into master Feb 1, 2023
@fmigneault fmigneault deleted the adapter-send-request branch February 1, 2023 15:47
fmigneault added a commit to bird-house/birdhouse-deploy that referenced this pull request Feb 10, 2023
…pie/weaver (#283)

Multiple updates to support secured WPS outputs.

## Changes

**Non-breaking changes**
- secure-data-proxy: add new [`secure-data-proxy`][secure-data-proxy] optional component.

  When enabled, this component will enforce authentication and authorization to be resolved against the `/wpsoutputs`
  endpoint prior to accessing the results produced by WPS executions. A Magpie service named `secure-data-proxy` is
  created to define the resource and permission hierarchy of directories and files the users and groups can access.
  When disabled, the original behavior to provide open access to `/wpsoutputs` is employed.

  A variable named `SECURE_DATA_PROXY_AUTH_INCLUDE` is dynamically assigned based on the activation or not of this
  component. Corresponding validation of optional/mandatory/delayed-eval variables used by this component are also
  applied dynamically, as well as mounting the necessary `nginx` and `docker-compose` extended configurations.

- Weaver: adjust user-context output directory hooks and permissions for [`secure-data-proxy`][secure-data-proxy].

  When a process defined in Weaver (either a WPS provider or a local definition) is executed by a user that was granted
  authorization to run a job, the corresponding user-context directory under `/wpsoutputs/users/{user-id}` will be used
  for storing the execution outputs and will have the appropriate permissions set for that user to grant them access to
  those outputs.

- Magpie/Twitcher: update minimum version `magpie>=3.31.0` to employ `twitcher>=0.8.0` in `MapgieAdatepr`.

  - Resolve an issue where `response.request` references were not set in OWS proxy responses when handled by Twitcher.
    This caused `MapgieAdatepr` response hooks to fail, which in turn caused failing requests for any non-WPS
    service that defined any proxy request hook, such as in the case of [`weaver`][weaver-component] component.

  - Adds the Twitcher ``/ows/verify/{service_name}[/{extra_path}`` endpoint employed for validating authorized access
    to Magpie service/resources, in the same fashion as the protected proxy endpoint, but without performing the proxied
    request toward the target service. This is mandatory for using the new [`secure-data-proxy`][secure-data-proxy] 
    optional component, otherwise the proxy endpoint triggers data download twice, once for authorization and another
    for actually accessing the data.

**Breaking changes**
- n/a (technically)
  New feature  [`secure-data-proxy`][secure-data-proxy] with optional component should default to using the original methodology of public access if not enabled.

## Related Issue / Discussion

- Depends on Ouranosinc/Magpie#571, which in turn depends on bird-house/twitcher#118. Corresponding Dockers `magpie>=3.31.0` and `twitcher>=0.8.0` must be deployed and used.
- Resolves [DAC-469 - *Secure WPS-Outputs served by Nginx*](https://crim-ca.atlassian.net/browse/DAC-469)
- Relates to [DAC-145 - *2.5.1 - Create private directory for new user for climate data [Cowbird/Thredds]*](https://crim-ca.atlassian.net/browse/DAC-145)
  (planing ahead some related permissions to user-context / wps outputs under weaver)

## Testing

To test the feature, simply enabled it in the `EXTRA_CONF_DIRS`, and place some files under `${DATA_PERSIST_ROOT}/wpsoutputs`. 
There should be a predefined `/wpsoutputs/public` location where files would be accessible when not logged in. Other locations should return 403. Then, a specific directory/file can be defined in Magpie with the relevant user permission on that resource to validate access to that file when logged in.

## To Do

- [x] Review and deploy Twitcher from bird-house/twitcher#118
- [x] Review and deploy Magpie from Ouranosinc/Magpie#571
- [x] Merge DACCS-config with `secure-data-proxy` branch for PR test suite

[secure-data-proxy]: birdhouse/optional-components/secure-data-proxy
[weaver-component]: birdhouse/components/weaver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants