add #[pymodule] mod some_module { ... }, v2 #3294
Conversation
|
Unfortunately now |
There was a problem hiding this comment.
Brilliant! Thank you for picking this up 🙏
I've put a comment below with a suggestion how we can avoid the test failure.
What do we need to finish here to make this useful? I think as a starting point it's good enough to merge if we can add submodules, classes and free functions with #[pymodule], #[pyclass] and #[pyfunction] items inside of the module. (Add them automatically instead of needing to add them in the #[module_init].)
We can add those to the test of test_declarative_module.
As follow ups we can deprecate #[pyfn] shorthand of functional modules, and maybe even completely deprecate functional modules (as #[module_init] should allow all the same functionality to be replicated).
| pub fn add_to_module(&'static self, module: &PyModule) -> PyResult<()> { | ||
| module.add_object(self.make_module(module.py())?.into()) | ||
| } |
There was a problem hiding this comment.
I think to avoid the reinitialization we could use a hypothetical self.find_or_make_module() here which uses PyState_FindModule to retrieve the existing module object (if it exists).
I think this exists on all CPython versions, for PyPy it only exists on PyPy 3.9 (with the wrong name) and PyPy 3.10. Probably it'll be good enough to just reinitialize on PyPy 3.8 and older, they're out of support anyway.
I'll send a PR now to fix the PyPy definitions for PyState_FindModule.
| item_fn.attrs.retain(|attr| { | ||
| let found = attr.path().is_ident("pymodule_init"); | ||
| is_module_init |= found; | ||
| !found | ||
| }); |
There was a problem hiding this comment.
I think this is where we can scan for #[pyclass], #[pyfunction] and #[pymodule].
If the implementation of how it would be cleanest to add these items to the module isn't readily available I'm happy to help figure this out.
| syn::UseTree::Glob(glob) => { | ||
| bail_spanned!(glob.span() => "#[pyo3] cannot import glob statements") | ||
| } |
There was a problem hiding this comment.
It would be nice to add a UI test for declarative modules which checks this error.
| !found | ||
| }); | ||
| if is_module_init { | ||
| ensure_spanned!(pymodule_init.is_none(), item_fn.span() => "only one pymodule_init may be specified"); |
There was a problem hiding this comment.
Similarly a UI test for this error here would be desirable.
| } = &mut module; | ||
| let items = match content { | ||
| Some((_, items)) => items, | ||
| None => bail_spanned!(module.span() => "`#[pymodule]` can only be used on inline modules"), |
There was a problem hiding this comment.
UI test also makes sense for this one, I think.
|
Following the mention in #3382, I hope this is a better place for suggesting that it would be great if PyO3 modules could have My motivation is that I'm working with non-standard architectures, where the process continues (so memory is not automatically released) but the interpreter is restarted, so I manually free all modules when the interpreter ends and reinitialize them (I'll be the first to admit that's a niche concern). |
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
|
Superseded by #3815 |
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
Based on PyO3#2367 and PyO3#3294 Allows to export classes, native classes, functions and submodules and provide an init function See test/test_module.rs for an example Future work: - update examples, README and guide - investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule] Co-authored-by: David Hewitt <mail@davidhewitt.dev> Co-authored-by: Georg Brandl <georg@python.org>
* #[pymodule] mod some_module { ... } v3
Based on #2367 and #3294
Allows to export classes, native classes, functions and submodules and provide an init function
See test/test_module.rs for an example
Future work:
- update examples, README and guide
- investigate having #[pyclass] and #[pyfunction] directly in the #[pymodule]
Co-authored-by: David Hewitt <mail@davidhewitt.dev>
Co-authored-by: Georg Brandl <georg@python.org>
* tests: group exported imports
* Consolidate pymodule macro code to avoid duplicates
* Makes pymodule_init take Bound<'_, PyModule>
* Renames #[pyo3] to #[pymodule_export]
* Gates #[pymodule] mod behind the experimental-declarative-modules feature
* Properly fails on functions inside of declarative modules
---------
Co-authored-by: David Hewitt <mail@davidhewitt.dev>
Co-authored-by: Georg Brandl <georg@python.org>
Implements using
#[pymodule]directly on modules.Allows importing other wrapped items with a
#[pyo3] use foo;.Allows having a
#[pymodule_init]function to run arbitrary code after adding all wrapped items.This is #2367 brought up to date with main.