Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(corner case) Ban negative indices #121

Closed
ManonMarchand opened this issue Dec 12, 2023 · 6 comments · Fixed by #149
Closed

(corner case) Ban negative indices #121

ManonMarchand opened this issue Dec 12, 2023 · 6 comments · Fixed by #149
Labels

Comments

@ManonMarchand
Copy link
Member

Due to the conversion from python int to rust u8 negative indices are not rejected and create false MOCs instead.

To solve this:

  • find every place where indices can be entered (ex: from_healpix_cells)
  • ban negative values on python side
@tboch
Copy link
Collaborator

tboch commented Jul 3, 2024

How does the conversion from python to rust type work?
Could this issue be solved on the Rust side?

@ManonMarchand
Copy link
Member Author

It silently accepts negative integers from python and converts them into unsigned integers in rust https://pyo3.rs/main/conversions/tables (which lead to crazy values)
I think it fits in this pyO3 issue but I'm not sure PyO3/pyo3#3226

Blanketing our integers before sending them to the rust side shouldn't affect the performance too much if it's your concern?

@tboch
Copy link
Collaborator

tboch commented Jul 3, 2024

My concern was rather about having a single checkpoint. I thought this could be managed on the Rust side, at conversion time, but it seems I'm wrong.

@ManonMarchand
Copy link
Member Author

maybe @fxpineau has an idea on how to do this better?

@fxpineau
Copy link
Member

fxpineau commented Jul 3, 2024

In which cases do we end up with negative HEALPix indices that are passed from Python to Rust?

Given that pyo3 do not produce and error (so far) when converting from a negative integer to an unsigned integer, we may consider at least two opinions:

  • prefer to have a single checkpoint on the Rust side: technically, I could accept signed integers on the Rust side, and then filter and cast into unsigned integer. Pros: no duplicated code / no risk to forget to check. Cons: 1. performance penalty for cases in which we are sure that all indices are ok (i.e. when checking is useless), 2. write code that would not exist if pyo3 behavior was to return an error;
  • prefer having a method checking (and cleaning) an array of HEALPix index on the Python side, and call it when necessary: we consider that the Rust method (accepting only unsigned integer) is a contract that must be fulfill, and the Python code calling the method has the responsibility to ensure it, identifying beforehand cases in which the contract may be broken (pros/cons are reversed with respect to the first bullet pros/cons).

I personally have a marked preference for the second bullet (check on Python side), possibly isolating and decorating with a check the call to the Rust method to mitigate the 'cons' (no duplication/no risk to forget, i.e. single checkpoint on the Python side).

@ManonMarchand
Copy link
Member Author

It was discovered due to the -1 that is used as padding in cdshealpix.neighbors when a cell has three neighbors instead of four. This generates a set of healpix's cells with spurious -1 among them.

I like FX's idea to define a decorator on the python's side. This way there is only one decorator to maintain and we can apply it to all methods that accept healpix cells (just as we already do with the validate_lonlat decorator that ensures that all coordinates sent to the rust side are in degrees, in icrs, and are wrapped)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants