v0.9rc2 testing (v0.9rc2-2-gc9c4b5e on OSX/Chrome) #129
Comments
bodom0015
changed the title
ResultsMajor: Test Failures (XXX)
Minor: Test Case Issues (TESTCASE)
PR for test plan improvements forthcoming |
|
PR for test plan improvements: #130 Outstanding Issues:
Open Questions:
|
|
|
Thank you for clarifying, @ThomasThelen I just reran the builds for Looking back, I'm not sure which was which but it looks like a From https://girder.stage.wholetale.org/#job/5e59902e351c4c0da4b48872: Step 51/76 : RUN R --quiet -e "install.packages('shiny', repos='https://mran.microsoft.com/snapshot/2020-02-26', method='libcurl')"
---> Running in 024c93a1c194
> install.packages('shiny', repos='https://mran.microsoft.com/snapshot/2020-02-26', method='libcurl')
�[91mInstalling package into ‘/srv/rlibs’
(as ‘lib’ is unspecified)
�[0m�[91malso installing the dependencies ‘BH’, ‘magrittr’, ‘httpuv’, ‘xtable’, ‘sourcetools’, ‘later’, ‘promises’, ‘fastmap’
... ... ... ... ... ... ... ... ... ... ... ...
CC src/libuv_la-fs-poll.lo
CC src/libuv_la-idna.loHTTPError: 502 Server Error: Bad Gateway for url: https://girder.stage.wholetale.org/api/v1/job/5e59902e351c4c0da4b48872
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 382, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/girder_worker/task.py", line 148, in __call__
results = super(Task, self).__call__(*_t_args, **_t_kwargs)
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 641, in __protected_call__
return self.run(*args, **kwargs)
File "/gwvolman/gwvolman/tasks.py", line 437, in build_tale_image
ret = _build_image(cli, tale_id, image, tag, temp_dir, REPO2DOCKER_VERSION)
File "/gwvolman/gwvolman/utils.py", line 324, in _build_image
print(line.decode('utf-8').strip())
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 158, in write
self._write_func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 259, in write
self._flush()
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 237, in _flush
req.raise_for_status()
File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)From https://girder.stage.wholetale.org/#job/5e598fb7351c4c0da4b48855: Step 57/64 : RUN chown -R ${NB_USER}:${NB_USER} ${REPO_DIR}
---> Running in 5cf07978e98a
Removing intermediate container 5cf07978e98aHTTPError: 502 Server Error: Bad Gateway for url: https://girder.stage.wholetale.org/api/v1/job/5e598fb7351c4c0da4b48855
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 382, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/girder_worker/task.py", line 148, in __call__
results = super(Task, self).__call__(*_t_args, **_t_kwargs)
File "/usr/local/lib/python3.5/dist-packages/celery/app/trace.py", line 641, in __protected_call__
return self.run(*args, **kwargs)
File "/gwvolman/gwvolman/tasks.py", line 437, in build_tale_image
ret = _build_image(cli, tale_id, image, tag, temp_dir, REPO2DOCKER_VERSION)
File "/gwvolman/gwvolman/utils.py", line 324, in _build_image
print(line.decode('utf-8').strip())
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 158, in write
self._write_func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 259, in write
self._flush()
File "/usr/local/lib/python3.5/dist-packages/girder_worker/utils.py", line 237, in _flush
req.raise_for_status()
File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self) |
50 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Test Plan
https://github.com/whole-tale/wt-design-docs/issues/new?template=test_plan.md
Note: For all tests, repeat for supported browser/OS combinations.
Preconditions:
Authentication
Navigation
Browse
Preconditions:
Assumes production Tales present (e.g., LIGO, materials, etc).
No running instances
General
TESTCASE: Search filtering appears to work, but the original LIGO Tale missing from master list on https://dashboard.stage.wholetale.org
TESTCASE: Add a step to "View Logs" while awaiting Tale image build
Manage
Preconditions
Empty home directory
No registered data
Register General
10.5065/D6862DM8XXX: Had to manually refresh after removing datasets - possibly whole-tale/dashboard#427?
XXX: Searching for this DOI/dataset yielded an error message:
No matching results found.Run
Preconditions:
No running Tale instances
General
TESTCASE: There is now only one option for exporting a Tale - wording could be adjusted slightly to convey this
Environmentdropdown menuLicensedropdownCreated by <your_name>underAuthorsPublished LocationreadsThis Tale has not been publishedTESTCASE: Missing confirmation of "Move To..." - how should this work?
XXX: Needed manual refresh in between copying and deleting - we should attempt to refresh the data after each operation (e.g. copy, delete, etc)
doi:10.18739/A29G5GD0V) on Manage > Data (see Manage > Register test cases above)XXX: Needed manual refresh in between copying and deleting - we should attempt to refresh the data after each operation (e.g. copy, delete, etc)
bag-info.txtbagit.txtdata/fetch.txtmanifest-md5.txtmanifest-sha256.txtmetadata/run-local.shtagmanifest-sha256.txttagmanifest-md5.txtREADME.mdmetadatafoldermanifest.jsonandenvironment.jsonfileLICENSEfileTESTCASE: I was not asked for confirmation, the download just started up
Settings
XXX: "Connect Account" should disable when no new resource_servers can be offered
XXX: "Connect Account" should disable when no new resource_servers can be offered
#370
/user/meendpoint to confirm tokens are still presentTale Creation
TESTCASE: Which Jupyter? All of them? (Current assumption is to try to run all of them)
XXX: I attempted to build all types of Jupyter Tales and received an error message (
Error: Building image) from bothJupyter NotebookandJupyter Classic- . I am unable to view the logs of the Tale to debug furtherCompose OpenRefine Tale
Too many instances
Analyze in Whole Tale
Task 1: Importing a Dataset from Dataverse
These test cases cover potential situations that can occur when importing datasets from Dataverse.
Testing Steps:
Replication Data for: "Agricultural Fires and Health at Birth"Input datasection matches the uri withData SourceappendedCreate New Tale and Launchbutton is disabledCreate New Tale and LaunchExternal DataTale WorkspacescienceTESTCASE: No "External Data" appeared, even after Tale was running. Dataset also did not appear under "Manage" after Tale was running (same as in #125). I believe this is a typo in the test case, as
READ/WRITEshouldn't result in any "External Data"Task 2: Importing a Dataset from DataONE
These test cases cover different scenarios that may happen when importing datasets from DataONE.
This test covers the ideal case where dataset importing has all of the needed information
to properly run. This includes the dataset location, title, and selected environment.
Testing Steps:
Fire influences on forest recovery and associated climate feedbacks in Siberian Larch Forests, RussiaSelected datasection matches the uri withData SourceappendedRstudiois selected in the Environments widgetCreate New Tale and Launchbutton is enabledCreate New Tale and LaunchscienceThe
Analyze in Whole Talefeature is deployed across a few DataONE test sites. This test ensures Whole Tale has the ability to importdata from each of them.
Links:
Testing Steps:
Create New Tale and LaunchTask 3: Shared Behavior
These cases show errors that are common between Dataverse and DataONE. Although they are tested using DataONE URIs, the errors should be the same when replacing it with a Dataverse URI.
This test covers the case where the name query parameter is missing. The expected behavior is
that the uri query parameter is used instead.
Testing Steps:
https://dataverse.harvard.edu/api/access/datafile/3323458Selected datasection matches the uri withData SourceappendedCreate New Tale and Launchbutton is disabledCreate New Tale and LaunchThis test covers the case where a user imports data from an external source and selects READ/WRITE in the Create Tale Modal.
Testing Steps:
Selected datasection matches the uri withData SourceappendedCreate New Tale and Launchbutton is disabledREAD/WRITECreate New Tale and LaunchTESTCASE: https://dashboard.stage.wholetale.org/browse?uri=https%3A%2F%2Fdataverse.harvard.edu%2Fapi%2Faccess%2Fdatafile%2F3323458 does not provide a title, so step 3 may not be accurate for that case
TESTCASE: https://dashboard.stage.wholetale.org/browse?api=https%3A%2F%2Fdev.nceas.ucsb.edu%2Fknb%2Fd1%2Fmn%2Fv2&environment=JupyterLab&name=proveit&uri=doi%3A10.5072%2FFK27P92Z55 provides an environment, so step 7 is not necessary for that case
XXX: https://dashboard.stage.wholetale.org/browse?api=https%3A%2F%2Fdev.nceas.ucsb.edu%2Fknb%2Fd1%2Fmn%2Fv2&environment=JupyterLab&name=proveit&uri=doi%3A10.5072%2FFK27P92Z55 yields only a
404.htmlin the Tale Workspace.. possibly a bug, but perhaps this is expected?Tale metadata tests
The purpose of these tests are to confirm that the metadata files (manifest.json, environment.json, LICENSE) we generate are correct.
manifest.jsonfile undermetadata/schema:authorcreatedByaggregatesaggregatesDatasetsmetadata/environment.jsonconfigsection from girder matches theconfigsection inenvironment.jsonLICENSEfileLICENSEfile matches the selected license on the metadata pageLICENSEchanged appropriatelyPublishing tests
Case 1: No Accounts Configured
Publish tale...Go to SettingsCase 2: Publishing to a Third Party
Connect to a third party provider (make sure it's a dev server)
Launch a Tale that you own
Add some data to it
Click
Publish tale...Confirm that the publishing modal opens
Confirm that the third party that you are connected to appears in the dropdown
Select the third party
Click
PublishOnce complete, navigate to the metadata page
Confirm that the published location is present
Visit the published location
Confirm that you see the following files plus yours
metadata.xmlLICENSEREADME.mdmanifest.jsonenvironment.jsonRepeat for each third party provider
a. After publishing to DataONE, use girder to query the Tale
b. Ensure that the 'pid' field in the 'publishInfo' object has an identifier in a doi format
XXX:
Zenodo Sandboxworked fine, butDataONE Devdid not - after fixing the description and the authors, I was unable to publish with the following error message:....... wat?
Published Locationin Run > metadataPublished Locationhas changed to the correct package landing pageZenodo integration tests
doi:10.5281/zenodo.16384ver_1https://zenodo.org/record/1172960#.XidIvlNKh24ver_2doi:10.5281/zenodo.1172960ver_2doi:10.5281/zenodo.608984ver_2The register tests the following cases.
Datasets with multiple files
Datasets with multiple versions
Concept DOI
Zenodo dataset via AiWT
The register tests the following cases.
Dataset with single file (zipfile)
Zenodo Binder via AiWT
publishInfofield on the tale object contains pid, repository, repository_id, uriTESTCASE: Cannot use the Water Tale for this, must create your own Tale -
Failed to get the deposition (id=490450). Server returned: 403 Client Error: FORBIDDEN for url: https://sandbox.zenodo.org/api/deposit/depositions/490450 { "message": "You don't have the permission to access the requested resource. It is either read-protected or not readable by the server.", "status": 403 }. Perhaps the imported tale already contains some publishInfo that is being reused?TESTCASE: Clarification needed here. I know how to export, but not sure I quite grasp how to "download and run local" - do you mean run the Docker image on my local machine?
Regression tests
/manageviewtest123) and click down into it/manageviewtest123folder and close the incognito tab/manageview of your Home folder/run/ThisIsObviouslyNotARealID/browsepage insteadI'll compile these down into a more-easily digestible list soon
The text was updated successfully, but these errors were encountered: