Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent xps.indices() from generating indices that flat index arrays #3184

Merged
merged 10 commits into from
Dec 10, 2021

Conversation

honno
Copy link
Member

@honno honno commented Dec 8, 2021

Resolves #3166.

I opted to simply filter the base extra._array_helpers.BasicIndexStrategy used inside of extra.array_api._indices(), which seems to do the trick... I didn't want to play around the internals haha. Let me know if you'd like a solution that doesn't filter itself. I'm fairly happy with the testing situation for indices now at least.

@Zac-HD
Copy link
Member

Zac-HD commented Dec 8, 2021

Hmm, I would rather avoid filtering here, to squeeze out all the performance we can - arrays are already fairly challenging for the Hypothesis internals. I'm happy to pick up that bit over the weekend if you're satisfied by the tests though 🙂

The failures are down to #3187, not anything you're doing.

@honno
Copy link
Member Author

honno commented Dec 9, 2021

Hmm, I would rather avoid filtering here, to squeeze out all the performance we can - arrays are already fairly challenging for the Hypothesis internals.

Ok I have a solution now that just disables the one area where flat indices are generated in BasicIndicesStrategy. Next time I should have a go before going the easy route 🙃

Copy link
Member

@Zac-HD Zac-HD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, with some minor comments about naming and comments. Your fix is in exactly the right place 😎

Logistics: rebase on master and CI should pass now (once you fix the coverage gap).

hypothesis-python/src/hypothesis/extra/_array_helpers.py Outdated Show resolved Hide resolved
hypothesis-python/src/hypothesis/extra/array_api.py Outdated Show resolved Hide resolved
hypothesis-python/tests/array_api/test_indices.py Outdated Show resolved Hide resolved
hypothesis-python/tests/array_api/test_indices.py Outdated Show resolved Hide resolved
@Zac-HD Zac-HD enabled auto-merge December 9, 2021 14:36
@@ -673,7 +685,7 @@ def do_draw(self, data):
while j < len(result) and result[j] == slice(None):
j += 1
result[i:j] = [Ellipsis]
else:
elif self.allow_fewer_indices_than_dims:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't actually have a test where this is False, so it might be time to copy the pre-edit test_indices_generate_valid_indexers over to our Numpy tests.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean True? I wrote a new test numpy/test_gen_data.py::test_basic_indices_can_generate_indices_not_covering_all_dims that checks flat indices are indeed actually generated, which parallels what array_api/test_indices.py/test_indices_generate_valid_indexers does to check they're not generated.

So I think in a logical sense I'm now covering both scenarios, but coverage is still failing. This has been a good excuse to finally start learning how to use coverage but yea haven't got round to fixing this today—let me know any quick fixes otherwise I'll explore this tomorrow.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake! I did mean False, even though there obviously are tests that execute it that way... they're just not run under coverage!

So slapping a # pragma: no branch on line 688 will fix this.

@Zac-HD Zac-HD merged commit 9ec4267 into HypothesisWorks:master Dec 10, 2021
@honno honno deleted the xp-filter-flat-indices branch February 28, 2024 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stop generating flat indices in xps.indices()
2 participants