-
Notifications
You must be signed in to change notification settings - Fork 722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PyMapping is inconsistent with typing.Mapping #2072
Comments
Yikes, thanks for reporting. So I'm happy to take the fall for this one. This is the C-API definition of PyMapping_Check:
Reading this again, it's such a broad definition that it doesn't really seem practically useful. And you're right that it's very different to On Python 3.10 and up we could potentially make use of However I'm not fond of having behaviour that's different according to the Python version. Alternatively, we could call into Python to pretty much literally do Ideas on a solution very welcome! |
I like the sound of using both the tp flags and the instance checks depending on your version |
I think this is awkward enough of a footgun (and also new enough, having been added in 0.15) that it would be good if we could come to a decision about what to do for this in time for the 0.16 release. All opinions very much wanted! |
Would be interesting to see what C extensions do. |
I think that we should consider trying to resolve this upstream in CPython. The proposed behavior (checking of tpflags/isinstance) feels like something that |
Remember guys, PyMapping_Check in CPython is used quite frequently and is meant to be as optimized as possible, while isinstance does not only check for inheritance. Modifying it in CPython will cause a discrepancy across versions, and since you're not actually checking a You can try blacklisting builtins but it's a whack a mole and will cause inconsistencies. All in all, unless you take The only possible solution is to maybe covert the ABCs and respective hook methods to C in order to optimize them, but even then it'll be much slower. |
I found some seemingly relevant discussion on BPO https://bugs.python.org/issue5945 One suggestion there is to use |
@aganders3 Nice find! I'll try to implement it and see how it behaves. |
See #2098 |
(from #2098 by @davidhewitt ) |
Created a bpo https://bugs.python.org/issue46376 |
It might decrease ergonomics, but maybe we should consider not converting mapping/sequences to the PyMapping/PySequence as the API is mostly for non-custom classes? |
Seems like |
It seems the upstream route is blocked, but maybe we can take the same out they did and just "fix" it in documentation. For example add a warning that downcast to a mapping may succeed for sequences and users may want to implement further checks as-needed. The |
I wouldn't call it blocked. I'm still working on a proposal to send to sig-capi and in the issue. |
Yeah that's fair I guess I wasn't sure if that was enough of a block to miss the 0.16 release this is targeted for. Also if it's a change going forward, does that still leave an open issue in Python <=3.10? |
I think the first comment from David is still the best approach |
Considering this is a wont-fix upstream, could we just implment Davids suggestion to fix this? |
I'm +1 on this and would be happy to implement as well if no one else wants to claim it. To be clear I think the proposal was:
|
Yes, I agree that we need to do something (current behavior seems broken). I still think the implementation above is likely to be ok. One possible complicating factor is that the flags aren't technically in the stable API - see python/cpython#32080 @samuelcolvin I think I spotted you used |
Thanks so much for making me aware. I hadn't seen this because 1) my tests for mappings were not good enough 🙈 2) I trusted pyo3 too much. Should be partially fixed in pydantic/pydantic-core#117 I would agree that your suggestion #2072 (comment) of the fast solution for 3.10 and falling back to python |
@aganders3 if you're willing to pick this up, would be very grateful. I think the only difference to my above proposal is that the fast path checking the flags would only apply on 3.10 and up when |
Hopefully you can continue to trust PyO3! This is the only huge footgun of a bug which I'm aware of 🤪 |
I'm running into a slight snag with this implementation for Python classes implemented in Rust. For 3.10+ I think we can add the This is causing a failure in the Any thoughts? |
Oh wait maybe I just need to add edit: nope, that would have to be implemented on the metaclass 😞 |
Hmm maybe for now you can just call Agreed #991 is the proper solution, I hope that progress can be made on that eventually... |
For people following this thread: in #2477 it looks like we've concluded that my proposal above isn't quite right and instead we should always do the equivalent of My question to you all: we are now wondering whether to change My inclination is yes, the consistency would be a highly desirable property. But it would be a breaking change so would like to take some time to consider it. |
Bug Description
The conversion table for PyMapping suggests anything that is a typing.Mapping should be able to be converted to PyMapping, this however is not the case seemly list and str are also downcast-able to a PyMapping.
This causes errors when you try to access things like items on the PyMapping as you get a AttributeError str has no attribute items.
Steps to Reproduce
Backtrace
No response
Your operating system and version
MacOS 11.1 (20C69)
Your Python version (
python --version
)3.9.3
Your Rust version (
rustc --version
)rustc 1.53.0-nightly (07e0e2ec2 2021-03-24)
Your PyO3 version
0.15.1
How did you install python? Did you use a virtualenv?
venv
Additional Info
No response
The text was updated successfully, but these errors were encountered: