Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve row checks in ListObjectsViewTestCase.test_list_objects_filtered #2580

Closed
glennmatthews opened this issue Oct 6, 2022 · 0 comments · Fixed by #3278
Closed

Improve row checks in ListObjectsViewTestCase.test_list_objects_filtered #2580

glennmatthews opened this issue Oct 6, 2022 · 0 comments · Fixed by #3278
Labels
type: housekeeping Changes to the application which do not directly impact the end user

Comments

@glennmatthews
Copy link
Contributor

Proposed Changes

ListObjectsViewTestCase.test_list_objects_filtered and ListObjectsViewTestCase.test_list_objects_unknown_filter_no_strict_filtering generic test cases both have logic like the following to check whether the object list view contains or doesn't contain various instances as expected:

            if hasattr(self.model, "name"):
                self.assertIn(instance1.name, content, msg=content)
                self.assertNotIn(instance2.name, content, msg=content)

This is fragile as it's scanning the entire HTML response for what may be a very small substring; as we move to have more and more of our test data defined by pseudo-random factories, including ones like name = factory.Faker("word", part_of_speech="noun") it's quite possible that generated test data may include text that coincidentally appears elsewhere in the HTML, causing false positive or false negative test results.

One present mitigation is to use capitalization in object names to reduce the likelihood of false matches (e.g. Red or RED is less likely to appear as a fragment of another word than red is) but more broadly it would be good to revisit this pattern and come up with a more robust way to look for specific rows in the table.

An approach I experimented with was to just look at the presence, absence, or count of <td class="even"> and <td class="odd"> substrings in the HTML instead, but that ended up failing for certain test cases such as the Job list view tests, since that view has custom HTML rendering.

Another possibility to consider would be moving these tests from unit to integration, since in integration tests we can use things like xpath lookups rather than naive substring matching...?

Justification

Improve test robustness.

@glennmatthews glennmatthews added the type: housekeeping Changes to the application which do not directly impact the end user label Oct 6, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: housekeeping Changes to the application which do not directly impact the end user
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant