Mephisto review testing suite #422
Conversation
…tes as well as review functionality. Review template and server have been adjusted to accomodate the testing suite.
facebook-github-bot
added
the
CLA Signed
Codecov Report
@@ Coverage Diff @@
## master #422 +/- ##
=======================================
Coverage 65.59% 65.59%
=======================================
Files 76 76
Lines 6970 6970
=======================================
Hits 4572 4572
Misses 2398 2398 Continue to review full report at Codecov.
|
What does the error message look like for this? Can you paste in the description? |
| This is great text, row4 | ||
| This is bad text, row4 | ||
| This is bad text, row5 | ||
| This is excellent text, row5 |
This is a portion of the output near the summary at the end. They key line of interest is the final one, as when it is printed jest does not exit by itself and the user must use ctrl-C to do so. |
|
I wonder if any of the solutions here could be useful: jestjs/jest#7287 Either calling a done() callback from the async, or using the |
I can try it out but I think once the timeout is surpassed the program never makes it to that part of the function. I was considering today reducing the timeout time for puppeteer and increasing the timeout on jest such that if every iteration on a test timedout they would remain within the jest timeout and then we could use expect statements to check whether puppeteer timed out or threw an error. |
|
FYI - Directions above should probably read: |
|
This looks great! Thanks for adding this testing suite into Mephisto. Excited to get it working as part of the GitHub build process in the future. This should also be a good foundational reference for those building upon these tests. You may have to resolve a few conflicts before merging this in, though I don't think they should be too bad. |
pringshia
changed the title
Overview
Mephisto review testing suite encorporates end to end testing for the mephisto review service. Testing suite tests for major UI elements on both the AllItemView and ItemView routes as well as the ability for the server to record reviews correctly.
Usage
PYTHON_PATHproperty on line 34 intests/test_constants.jsto match that pathmkdir test/integration/review/test_app/Mephisto:npx create-react-app test-review --template file:packages/cra-template-mephisto-reviewcd test-reviewnpm run build/Mephisto/test/integration/review/test_app/build:cp -r build ../test/integration/review/test_app/buildcd ../test/integration/reviewnpm testtests/test_constants.jsfile. This is the time to wait for the server to start and stop. Tests may fail by default on certain devices because these values are too small and the server does not start within the time the tests expect it to. Change as necessary to make sure server has enough time to start and stop.Key files
Behaviour
jest --runInBand --verbose--runInBandoption is used because each test requires interacting with the server on an individual level, this flag allows each test to run independent of each other..test.jsfilesstarts child process with given arguments, waits delay_time milliseconds afterwards for server to setup, returns true if server started successfully, false if not.
server_start_argsis a mandatory argument, must be an array of arguments meant to be used in a node.jsspawnfunction which can start a mephisto review server.data_collection_argsis an optional argument, must be an array of arguments for a Node.jsspawnwhich creates standard output that can be piped into the standard input of mephisto review server process to provide the server with data.delay_in_msis a mandatory argument, must be an integer that represents how long in milliseconds the function waits after creating the review server to test a successful start.expected_server_outputis a mandatory argument, must be a string that represents the expected standard output from the mephisto review server upon a successful start.stdoutCallbackis an optional argument for a callback that can be attached to the review server's standard output stream.stdCalloutmust be a function and will receive a data buffer as a parameter.kills child process review server, returns true if successfully killed, false if not.
delay_in_msrepresents how long function waits before checking if server was successfully killed. Recommended that delay_in_ms is at least 500ms.typeparameter can be"CSV"ornullto indicate special file parsing behaviour. File lines are loaded into the array with line number corresponding to the array index with 0-numbered lines. METHOD SHOULD ONLY BE USED FOR SMALL TEXT FILES.uses server.js to start a review server and puppeteer to navigate to the root route of the review server. Checks if all major UI elements are present on AllItemView. Checks include search bar functionality and the presence of all given data points with a unique ID identifier. Checks are performed for single page and multiple pages of both CSV and JSON data. Then proceeds to stop the child process.
uses server.js to start a review server and then uses puppeteer to navigate to the root route of the review server. For each data card present on the review server, navigates to the ItemView route of the data card. Checks if all major UI elements are present on the ItemView. Checks include presence of correct data point as well as unique ID identifier as well as the availability of the two default review buttons. Checks are performed for single page and multiple pages of both CSV and JSON data. Then proceeds to stop the child process.
uses server.js to start a review server. Uses puppeteer to navigate to root route of review app. Navigates to each ItemView for each data point. Approves or Rejects each data point depending on the test. Proceeds to stop review server. Review data ia logged in standard output from the review server. Matches recorded review output to the performed review on each data point. Checks are performed for single page and multiple pages of both CSV and JSON data.
TODO
Issues