Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in cpp iteration over rvalue-dependant attribute #3828

Merged
merged 5 commits into from
Aug 3, 2022

Conversation

xavth
Copy link
Contributor

@xavth xavth commented Sep 14, 2020

The implementation of the Python-style for ... in loop over an iterable cpp sequence (using begin() and end()) needs to determine whether the sequence should be stored in a temporary variable or if it can be iterated over directly.

The previous code assumed that attributes are always safe to be iterated directly, but attributes can depend on an rvalue, as in:

for i in returns_object_with_vector_attribute().vector:
    ...

In such a case we need to store the in operand in a temporary value on which begin() and end() can be safely called.

It might make sense to move the test into tests/run/cpp_iterators

@xavth
Copy link
Contributor Author

xavth commented Sep 14, 2020

Should I open an issue ? Should I copy the commit message to the PR description ?

@da-woods
Copy link
Contributor

Should I open an issue ? Should I copy the commit message to the PR description ?

Definitely copy the commit message here so that the PR makes sense. I'm not convinced that an issue is really needed but others might disagree.

@xavth
Copy link
Contributor Author

xavth commented Sep 14, 2020

Should I open an issue ? Should I copy the commit message to the PR description ?

Definitely copy the commit message here so that the PR makes sense. I'm not convinced that an issue is really needed but others might disagree.

OK, I updated the PR description with the description in the commit message.

The implementation of the Python-style `for ... in` loop over an
iterable cpp sequence (using `begin()` and `end()`) needs to determine
whether the sequence should be stored in a temporary variable or
if it can be iterated over directly.

The previous code assumed that attributes are always safe to be
iterated directly, but attributes can depend on an rvalue, as in:

    for i in returns_object_with_vector_attribute().vector:
        ...

In such a case we need to store the `in` operand in a temporary value
on which `begin()` and `end()` can be safely called.

It might make sense to move the test into tests/run/cpp_iterators
@xavth
Copy link
Contributor Author

xavth commented Sep 15, 2020

I noticed that the test builds on OSX seemed to fail because I used C++11-style list-initialization in the external C++ code to support my testcase, so I modified it.

The implementation of the Python-style `for ... in` loop over an
iterable cpp sequence (using begin() and end()) needs to determine
whether the sequence should be stored in a temporary variable or
if it can be iterated over directly.

A previous fix considers that the sequence can be accessed directly
if `sequence.is_simple()` evaluates to True.

However it is at least theoretically be possible for an expression
to both be simple and to depend on temporary subexpressions. In that
case, because the CppIteratorNode is always a temporary, the compiler
will by default eagerly free the temporary subexpressions for reuse
after the result of the sequence has been evaluated the first time.
This will result in subsequent calls to `result()` returning `None`
even though the required variables are still available and in scope
in the generated C++ code.

This commit postpones the release of temporary subexpressions until
the CppIteratorNode is itself disposed-of when the sequence is
accessed directly.
@da-woods
Copy link
Contributor

Just closing and re-opening to re-run the tests (just trying to see if I can clear up some old inactive PRs)

@da-woods da-woods closed this Jun 26, 2022
@da-woods da-woods reopened this Jun 26, 2022
@da-woods da-woods added the C++ label Jun 26, 2022
@da-woods
Copy link
Contributor

I've moved the tests into the main cpp_iterators file (but made no other changes to them).

I think I'm happy with this PR and would like to merge it. I'll wait for a bit to see if there's other comments/objections though

@da-woods
Copy link
Contributor

I'd been planning to merge this after the next release - I think it's a worthwhile (if slightly obscure) bug fix (link back to #4022)

@da-woods da-woods merged commit 287a11e into cython:master Aug 3, 2022
@da-woods
Copy link
Contributor

da-woods commented Aug 3, 2022

Thanks, and sorry it took a while to merge

@xavth
Copy link
Contributor Author

xavth commented Aug 18, 2022

Thanks for merging @da-woods :) and sorry in turn I didn't answer sooner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants