-
Notifications
You must be signed in to change notification settings - Fork 40
Test case creation: add helper function to create test case #1017
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Nitish Bharambe <nitish.bharambe@alliander.com>
Signed-off-by: Nitish Bharambe <nitish.bharambe@alliander.com>
Co-authored-by: Martijn Govers <martijn.govers@alliander.com> Signed-off-by: Nitish Bharambe <78108900+nitbharambe@users.noreply.github.com>
Signed-off-by: Nitish Bharambe <nitish.bharambe@alliander.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nitbharambe @mgovers
My proposal is to move the validation test function (not the test case finder and parametrize) to the PGM package.
The validation test function should read a path to a test case, and some meta data (like calculation method, calculation type, etc.). And it should run the validation tests.
In this way, user can easily run more validation test just by install PGM self without going into developer mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adds a new helper _make_test_case
to generate and serialize end-to-end test artifacts (input, optional update batch, output, params) along with license files, and provides unit tests to verify its behavior.
- Introduce
LICENSE_TEXT
constant and_make_test_case
function inutils.py
. - Update
test_utils.py
to import the helper and addtest__make_test_case
covering batch and non-batch flows. - Ensure serialization and license-writing calls are made for each generated file.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
src/power_grid_model/utils.py | Add LICENSE_TEXT and implement _make_test_case to write JSON and license files. |
tests/unit/test_utils.py | Import LICENSE_TEXT , DatasetType , _make_test_case ; add parametrized test for the new helper. |
Comments suppressed due to low confidence (3)
tests/unit/test_utils.py:208
- Add an assertion to verify that
params.json
itself is written (e.g.,write_text_mock.assert_any_call(save_path / "params.json", data=json.dumps(params, indent=2), encoding="utf-8")
) to ensure full coverage of all output files.
write_text_mock.assert_any_call(save_path / "params.json.license", data=LICENSE_TEXT, encoding="utf-8")
tests/unit/test_utils.py:185
- [nitpick] Consider renaming
test__make_test_case
totest_make_test_case
for consistency with typical pytest naming conventions.
def test__make_test_case(
tests/unit/test_utils.py:172
- Add a negative test case to cover invalid
output_dataset_type
and assert that it raisesValueError
to fully test the error path in_make_test_case
.
@pytest.mark.parametrize(
While indeed useful, I think that's a separate feature that is independent of this PR. In the first place, we want users to be able to provide us with something that is debugable (a test case). If we want to allow them to debug themselves by publishing the test loader, then we should do some serious cleanup work on it first. We would also need to decide whether to publish the test data as well (or maybe a subset of the data), and a design on how to do that without making the PGM package too big |
I think this is the same feature from user-story perspective. If user has the function to create a validation dataset, the user wants to be able to run the validation without installing PGM in developer mode or cloning the source code (including the test scripts). So from user perspective merging only this function alone does not make sense. |
Still, i would argue that running validation cases is a separate PR at the minimum. I think it's good to follow an incremental improvement approach here. I think cleaning up the test case runner needs to be prioritized with the team and would cause needless delays for merging this PR |
Signed-off-by: Nitish Bharambe <nitish.bharambe@alliander.com>
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test runner into package will be done in a separate PR. I hereby approve this PR.
Changes proposed in this PR include:
A helper function is added to make adding validation test cases easier.