Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

catalog_search.ipynb fails on Jenkins for staging PAVICS server #147

Closed
tlvu opened this issue Jan 9, 2020 · 1 comment · Fixed by #181
Closed

catalog_search.ipynb fails on Jenkins for staging PAVICS server #147

tlvu opened this issue Jan 9, 2020 · 1 comment · Fixed by #181

Comments

@tlvu
Copy link
Contributor

tlvu commented Jan 9, 2020 •
edited
Loading

catalog_search.ipynb fails on Jenkins simply because the staging PAVICS server (Medus) do not have the same data on Thredds as the production PAVICS server.

Could we re-write the search query to query data that we know exist on both servers? Like data under folder testdata?

Or we have to document the minimum dataset required on all PAVICS servers for tests to pass. This minimum dataset should not be too big so all PAVICS deployments for all different organizations can also deploy them to be alble to run the same Jenkins test suite against their PAVICS deployment.

Currently this is the only Jenkins test failing on staging servers !

Jenkins error

11:31:38  _____ pavics-sdi-master/docs/source/notebooks/catalog_search.ipynb::Cell 1 _____
11:31:38  Notebook cell execution failed
11:31:38  Cell 1: Cell outputs differ
11:31:38  
11:31:38  Input:
11:31:38  resp = wps.pavicsearch(constraints="variable:tasmin,project:CMIP5,experiment:rcp85,frequency:day", limit=10, type="File")
11:31:38  [result, files] = resp.get(asobj=True)
11:31:38  files
11:31:38  
11:31:38  Traceback:
11:31:38   mismatch 'text/plain'
11:31:38  
11:31:38   assert reference_output == test_output failed:
11:31:38  
11:31:38    "['https://me...21001231.nc']" == '[]'
11:31:38    + []
11:31:38    - ['https://medus.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/CCCMA/CanESM2/rcp85/day/atmos/r5i1p1/tasmin/tasmin_day_CanESM2_rcp85_r5i1p1_20060101-21001231.nc',
11:31:38    -  'https://medus.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/CCCMA/CanESM2/rcp85/day/atmos/r2i1p1/tasmin/tasmin_day_CanESM2_rcp85_r2i1p1_20060101-21001231.nc',
11:31:38    -  'https://medus.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/CCCMA/CanESM2/rcp85/day/atmos/r3i1p1/tasmin/tasmin_day_CanESM2_rcp85_r3i1p1_20060101-21001231.nc',
11:31:38    -  'https://medus.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/CCCMA/CanESM2/rcp85/day/atmos/r4i1p1/tasmin/tasmin_day_CanESM2_rcp85_r4i1p1_20060101-21001231.nc',
11:31:38    -  'https://medus.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/CCCMA/CanESM2/rcp85/day/atmos/r1i1p1/tasmin/tasmin_day_CanESM2_rcp85_r1i1p1_20060101-21001231.nc']

Edit:

  • add Jenkins error
@tlvu tlvu mentioned this issue Jan 29, 2020
Closed
@tlvu
Copy link
Contributor Author

tlvu commented Feb 16, 2020

@huard @tlogan2000 can you guys suggest a minimum and small test dataset to be include in all Thredds so the catalog_search.ipynb notebook works for all PAVICS deployment, not just our production one?

Preferably the same dataset as in bird-house/birdhouse-deploy#6 so it's one stone two birds.

@tlvu tlvu mentioned this issue Oct 30, 2020
Merged
@tlvu tlvu closed this as completed in #181 Nov 3, 2020
tlvu added a commit that referenced this issue Nov 3, 2020
@tlvu
Merge pull request #181 from Ouranosinc/nbval-ignore-catalog_search
2af1be4 
catalog_search.ipynb: rewrite query so the data needed on Thredds server is minimum

Fixes #147

This notebook is a good round trip testing for
* crawler can crawl data on Thredds
* crawler can insert proper data in Solr DB
* catalog search is able to query Solr DB to retrieve the expected data

With the query requiring only the minimum data on Thredds that was already needed by other notebooks, this notebook should pass on all deployments of PAVICS given the minimum test data is available and the crawler has been triggered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@tlvu