Skip to content

Add instructions on how to run regress tests and how to audit a new regress tests #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
safern opened this issue Mar 26, 2025 · 2 comments
Assignees
Milestone

Comments

@safern
Copy link
Member

safern commented Mar 26, 2025

We should add a documentation on how to run and audit regress tests. Also a brief overview of how they work.

Here's a summary of how to audit a new test (this needs to be cleaned up):

This is a regress test for example:
https://github.com/microsoft/documentdb/blob/main/pg_documentdb/src/test/regress/sql/commands_delete.sql

You need to add a new sql file on that directory. Then add it to the schedule here:
https://github.com/microsoft/documentdb/blob/main/pg_documentdb/src/test/regress/basic_schedule

You can run the individual test with:

make check-minimal -C pg_documentdb EXTRA_TESTS=<test-file-without-extension

You will see a test failure, because there is no expected .out file, so an empty one will be created in this directory:
https://github.com/microsoft/documentdb/tree/main/pg_documentdb/src/test/regress/expected

If the output of your test looks as expected, copy the contents from:
pg_documentdb/src/test/regress/results/<your_test.out> into the expected .out file.

To run all tests just do:

make check

To get the next collection id to use at the top of your test file after running make check you can run this script:
https://github.com/microsoft/documentdb/blob/main/pg_documentdb/src/test/regress/validate_test_output.sh

@safern safern added this to the 0.103-0 milestone Mar 26, 2025
@lichoil
Copy link

lichoil commented Apr 23, 2025

@SelinaTang21 will be working on this documentation.

@SelinaTang21
Copy link

I’ve added a preliminary guide based on the procedure @safern provided! https://github.com/microsoft/documentdb/wiki/Regression-Tests-Guide Waiting for PM's review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants