Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update nb for wpsoutputs path change to bird name #280

Merged
merged 2 commits into from Feb 17, 2023

Conversation

tlvu
Copy link
Contributor

@tlvu tlvu commented Feb 15, 2023

See PR bird-house/birdhouse-deploy#203

Jenkins failure due to wpsoutputs path change:

  ___ pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb::Cell 23 ___
  Notebook cell execution failed
  Cell 23: Cell outputs differ

  Input:
  from urllib.parse import urlparse

  output_url = result.get().output
  print("output_url = ", output_url)
  parsed = urlparse(output_url)
  output_path = parsed.path.replace("wpsoutputs", "wps_outputs")
  print("output_path = ", output_path)

  output_thredds_url = (
      f"https://{parsed.hostname}/twitcher/ows/proxy/thredds/dodsC/birdhouse{output_path}"
  )
  print("output_thredds_url = ", output_thredds_url)

  Traceback:
   mismatch 'stdout'

   assert reference_output == test_output failed:

    'output_url =...FILE/out.nc\n' == 'output_url =...FILE/out.nc\n'
    - output_url =  https://WPS_HOST/wpsoutputs/finch/STATUS_FILE/out.nc
    ?                                           ------
    + output_url =  https://WPS_HOST/wpsoutputs/STATUS_FILE/out.nc
    - output_path =  /wps_outputs/finch/STATUS_FILE/out.nc
    ?                            ------
    + output_path =  /wps_outputs/STATUS_FILE/out.nc
    - output_thredds_url =  https://PAVICS_FQDN/twitcher/ows/proxy/thredds/dodsC/birdhouse/wps_outputs/finch/STATUS_FILE/out.nc
    ?                                                                                                  ------
    + output_thredds_url =  https://PAVICS_FQDN/twitcher/ows/proxy/thredds/dodsC/birdhouse/wps_outputs/STATUS_FILE/out.nc

  _______ pavics-sdi-master/docs/source/notebooks/subsetting.ipynb::Cell 4 _______
  Notebook cell execution failed
  Cell 4: Cell outputs differ

  Input:
  res = resp.get()
  print("URL: ", res.output)
  res = resp.get(asobj=True)
  res.output

  Traceback:
   mismatch 'stdout'

   assert reference_output == test_output failed:

    'URL:  https:..._Africa.nc.\n' == 'URL:  https:..._Africa.nc.\n'
    - URL:  https://WPS_HOST/wpsoutputs/flyingpigeon/STATUS_FILE/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc
    ?                                  -------------
    + URL:  https://WPS_HOST/wpsoutputs/STATUS_FILE/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc
      Downloading to /tmp/tmpRANDOM/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc.

See PR bird-house/birdhouse-deploy#203

Jenkins failure due to wpsoutputs path change:
```
  ___ pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb::Cell 23 ___
  Notebook cell execution failed
  Cell 23: Cell outputs differ

  Input:
  from urllib.parse import urlparse

  output_url = result.get().output
  print("output_url = ", output_url)
  parsed = urlparse(output_url)
  output_path = parsed.path.replace("wpsoutputs", "wps_outputs")
  print("output_path = ", output_path)

  output_thredds_url = (
      f"https://{parsed.hostname}/twitcher/ows/proxy/thredds/dodsC/birdhouse{output_path}"
  )
  print("output_thredds_url = ", output_thredds_url)

  Traceback:
   mismatch 'stdout'

   assert reference_output == test_output failed:

    'output_url =...FILE/out.nc\n' == 'output_url =...FILE/out.nc\n'
    - output_url =  https://WPS_HOST/wpsoutputs/finch/STATUS_FILE/out.nc
    ?                                           ------
    + output_url =  https://WPS_HOST/wpsoutputs/STATUS_FILE/out.nc
    - output_path =  /wps_outputs/finch/STATUS_FILE/out.nc
    ?                            ------
    + output_path =  /wps_outputs/STATUS_FILE/out.nc
    - output_thredds_url =  https://PAVICS_FQDN/twitcher/ows/proxy/thredds/dodsC/birdhouse/wps_outputs/finch/STATUS_FILE/out.nc
    ?                                                                                                  ------
    + output_thredds_url =  https://PAVICS_FQDN/twitcher/ows/proxy/thredds/dodsC/birdhouse/wps_outputs/STATUS_FILE/out.nc

  _______ pavics-sdi-master/docs/source/notebooks/subsetting.ipynb::Cell 4 _______
  Notebook cell execution failed
  Cell 4: Cell outputs differ

  Input:
  res = resp.get()
  print("URL: ", res.output)
  res = resp.get(asobj=True)
  res.output

  Traceback:
   mismatch 'stdout'

   assert reference_output == test_output failed:

    'URL:  https:..._Africa.nc.\n' == 'URL:  https:..._Africa.nc.\n'
    - URL:  https://WPS_HOST/wpsoutputs/flyingpigeon/STATUS_FILE/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc
    ?                                  -------------
    + URL:  https://WPS_HOST/wpsoutputs/STATUS_FILE/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc
      Downloading to /tmp/tmpRANDOM/tasmax_Amon_MPI-ESM-MR_rcp45_r1i1p1_200601-200612_Africa.nc.
```
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Contributor

Nice!

@@ -90,7 +90,7 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #1.    ds.heat_wave_frequency.plot()

I'm assuming the changing memory address is not picked up by the output diff during E2E tests?


Reply via ReviewNB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a regex to ignore those memory address change.

tlvu added a commit to bird-house/birdhouse-deploy that referenced this pull request Feb 17, 2023
Before this fix, finch, raven, flyingpigeon were dumping their output
directly under `https://PAVICS_HOST/wpsoutputs/`.

With this fix, it will be under each bird name, ex:
`https://PAVICS_HOST/wpsoutputs/finch/` which is cleaner and follows
what malleefowl and hummingbird already does.

Fixes #11.
Fixes https://crim-ca.atlassian.net/browse/DAC-398

Requires PR Ouranosinc/pavics-sdi#280,
bird-house/finch#273,
Ouranosinc/raven#459

If `optional-components/secure-data-proxy` is enabled, might need some
additional permissions for each bird in

https://github.com/bird-house/birdhouse-deploy/blob/master/birdhouse/optional-components/secure-data-proxy/config/magpie/config.yml.template.
@tlvu tlvu merged commit f4aecf6 into master Feb 17, 2023
@tlvu tlvu deleted the fix-nb-for-wpsoutput-path-change branch February 17, 2023 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants