Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-46376: add (failing) tests for ctypes/pointer/cast/set_contents #108519

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

code-of-kpp
Copy link
Contributor

Previous attempt to fix gh-46376 was incomplete and overall it didn't succeed, and was reverted. However, we have discovered some dangerous issues with ctypes, that aren't fixed or documented anywhere.
This commit adds tests (@expectedfailure) so at least developers are aware of situations where memory might be corrupted, leaked or when changing a pointer value might have no effect.
Hopefully, we should be able to remove @expectedfailure in the future, with new shiny ctypes implementation or at least a bugfix.

/cc @ambv

@ambv
Copy link
Contributor

ambv commented Aug 26, 2023

The previous fix was only reverted for 3.11 so we could release 3.11.5. The rest isn't in such a hurry so I hope we can fix it forward instead.

@code-of-kpp
Copy link
Contributor Author

code-of-kpp commented Aug 26, 2023

I'm planning to fix it yes, but I'd like the tests to be merged first, if you agree that it's reasonable

@code-of-kpp code-of-kpp force-pushed the test-p-set-contents branch 3 times, most recently from 5cf867d to a84b664 Compare August 26, 2023 19:18
@code-of-kpp
Copy link
Contributor Author

There are too many moving parts:

  1. my assumption that I can always find original object in "1"'s key in _objects is simply wrong, e.g., if the pointer is inside a structure, or it's one of the pointers in an array of pointers
  2. your attempt to fetch _objects directly also wouldn't work, for the same reason: you need to follow the chain of _b_base to find the shared _objects.
  3. it's not only Pointer_set_contents/Pointer_get_contents, but apparently Array_set.../Array_get, and probably PyCField_get/PyCField_set, that can create pointer clones.
  4. It's OK for different pointers (who have own memory address themselves), to point to the same object, but it's really difficult to ensure that they'll share _objects.
    ... and probably I'm missing something else

The biggest bug right now seems to be that set_contents might change the address of the pointer, not just the contents. That one, probably, we can fix. Still the solution for it wouldn't be what I originally proposed.

Another shortcut, would be to take my original idea, look into _objects["1"], make sure it's actually the same object and only in this case – return it. (so we'll have a partial fix).

@code-of-kpp code-of-kpp force-pushed the test-p-set-contents branch 2 times, most recently from 3e346a8 to 5dbb4ae Compare August 27, 2023 22:51
@code-of-kpp code-of-kpp force-pushed the test-p-set-contents branch 2 times, most recently from 972c50f to d44a269 Compare September 6, 2023 23:43
Previous attempt to fix pythongh-46376 was incomplete and
overall it didn't succeed, and was reverted in 3.11.
However, we have discovered some dangerous issues with
ctypes, that aren't fixed or documented anywhere.
This commit adds tests (@expectedfailure) so at least
developers are aware of situations where memory might
be corrupted, leaked or when changing a pointer value
might have no effect.
Hopefully, we should be able to remove @expectedfailure
in the future, with new shiny ctypes implementation or
at least a bugfix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ctypes pointer not always keeping target alive
4 participants