Skip to content

Conversation

crusaderky
Copy link
Contributor

@github-actions github-actions bot added scipy.stats scipy.optimize scipy.signal scipy.integrate scipy._lib scipy.fft array types Items related to array API support and input array validation (see gh-18286) maintenance Items related to regular maintenance tasks labels Jan 13, 2025
@mdhaber
Copy link
Contributor

mdhaber commented Jan 13, 2025

I had assumed the purpose of using un-wrapped namespaces in the tests was for the tests to better reflect the conditions in practice, assuming end-users do not use array_api_compat.

That said, since array_api_compat does not wrap array objects themselves (IIRC), it seems that the SciPy function being tested will behave the same way whether the arrays were generated using a wrapped or unwrapped namespace.

From that perspective, providing a wrapped xp to the unit tests really shouldn't affect the results , and it would make writing the tests more convenient. So if that's right, seems good to me.

@crusaderky
Copy link
Contributor Author

crusaderky commented Jan 13, 2025

That said, since array_api_compat does not wrap array objects themselves (IIRC)

Correct.

I don't know if the SciPy function being tested will behave any differently whether the array arguments were generated using a wrapped or unwrapped namespace.

Unless a user-facing function accepts xp as an optional parameter and the test explicitly passes xp=xp to it, it can't behave any different. I haven't witnessed this specific condition throughout the scipy codebase.
Another edge case would be for a scipy function to accept a callback that points back into the namespace.

@mdhaber
Copy link
Contributor

mdhaber commented Jan 13, 2025

Unless a user-facing function accepts xp as an optional parameter

I am not aware of any.
So +1 for the idea.

@lucascolley
Copy link
Member

I think this works, yes. I suppose if array-api-compat ever did modify the array objects then we would have problems.

Unless a user-facing function accepts xp as an optional parameter

https://docs.scipy.org/doc/scipy/reference/generated/scipy.fft.fftfreq.html

@crusaderky
Copy link
Contributor Author

I think this works, yes. I suppose if array-api-compat ever did modify the array objects then we would have problems.

Unless a user-facing function accepts xp as an optional parameter

https://docs.scipy.org/doc/scipy/reference/generated/scipy.fft.fftfreq.html

You also need a test that explicitly passes the unwrapped xp, otherwise it will default internally to array_namespace(args)

Comment on lines 584 to 585
assert xp_size(xp.unique_values(res.nit)) == 4
assert xp_size(xp.unique_values(res.nfev)) == 4
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fails in CI PyTorch, stating that res.nit has 3 unique elements. Can't reproduce locally.
https://github.com/scipy/scipy/actions/runs/12767945483/job/35587336176?pr=22320

To me it seems like a genuine bug but I don't have enough domain knowledge to understand what's right and wrong. @mdhaber could you have a look?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a bug. I'll fix the test in a bit . You can skip that line if it's holding you up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skipped, ty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking more carefully, this part of the test was NumPy-only before the PR:

if is_numpy(xp):
assert len(np.unique(res.nit)) == 4
assert len(np.unique(res.nfev)) == 4

The purpose is to check that it's possible for all four numbers to be different. It was challenging to find an example for which that was the case, and it turned out to be sensitive to arithmetic, so we made it NumPy only.

It must have started failing only because the if is_numpy line was removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It must have started failing only because the if is_numpy line was removed?

Correct. I find surprising that different backends can yield different results.

@crusaderky
Copy link
Contributor Author

Ready for review.
See comment above about the CI failure.

@crusaderky crusaderky marked this pull request as ready for review January 14, 2025 15:11
@crusaderky crusaderky changed the title WIP TST: Array API: wrap namespaces centrally TST: Array API: wrap namespaces centrally Jan 14, 2025
@lucascolley lucascolley changed the title TST: Array API: wrap namespaces centrally TST: array types: wrap namespaces centrally Jan 14, 2025
Copy link
Member

@lucascolley lucascolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @crusaderky, very nice clean-up

@lucascolley lucascolley added this to the 1.16.0 milestone Jan 14, 2025
crusaderky and others added 2 commits January 14, 2025 16:58
Co-authored-by: Lucas Colley <lucas.colley8@gmail.com>
@lucascolley lucascolley merged commit 564da79 into scipy:main Jan 14, 2025
37 of 39 checks passed
@crusaderky crusaderky deleted the xp_test branch January 14, 2025 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
array types Items related to array API support and input array validation (see gh-18286) maintenance Items related to regular maintenance tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants