Skip to content

Conversation

@itamaro
Copy link
Contributor

@itamaro itamaro commented Nov 16, 2025

I still don't know what's the context behind this assertion, just that I didn't see any issues from removing it.

with this, test_embed fails with
```
Assertion failed: (PyUnicode_FindChar(name, '.', 0, PyUnicode_GetLength(name), -1) == -1), function _Py_ext_module_loader_info_init_for_builtin, file importdl.c, line 159.
```
with this, test_embed succeeds again
@encukou
Copy link
Member

encukou commented Nov 17, 2025

It was added in #118204

@ericsnowcurrently, do you remember if there was a reason for this assert?
Can we assume you added asserts when exploring which code paths are taken, and since we had no tests for modules with a dot added via PyImport_AppendInittab, this assert just never fired?


@itamaro: I'd be a bit more careful in _testembed: from test import create_static_module assumes that the test package is available, which is not always the case. I don't think you'd necessarily want to run testembed without test, but I also don't think we should add an arbitrary requirement/assumption.

Could you add a different containing package to the inittab?

@brettcannon brettcannon removed their request for review November 17, 2025 19:24
@itamaro
Copy link
Contributor Author

itamaro commented Nov 18, 2025

@itamaro: I'd be a bit more careful in _testembed: from test import create_static_module assumes that the test package is available, which is not always the case. I don't think you'd necessarily want to run testembed without test, but I also don't think we should add an arbitrary requirement/assumption.

Could you add a different containing package to the inittab?

Can _testembed even be meaningfully used without the test package? (and test_embed.py driving it)

Regardless, I agree this is an awkward way to test this. I think it would make more sense to add a net new test case for this functionality, if we decide it's a desired functionality we don't want to regress.
Happy to do that when we reach such a conclusion, but let's first make sure the assertion is safe to remove!

@encukou
Copy link
Member

encukou commented Nov 18, 2025

I'd say that as an embedder, you're probably in the best position both to test this, and to decide if it's desired functionality.

Allowing submodules in inittab looks doable. (@brettcannon, do you have any thoughts on BuiltinImporter loading submodules?)

Allowing packages, on the other hand, might be worth more discussion -- that would probably need new API, since struct _inittab doesn't have the is_package bit.

Can _testembed even be meaningfully used without the test package? (and test_embed.py driving it)

It can! Not sure if it's meaningful to run it without the test package available, but I do use the CLI on its own when debugging.

Anyway that's not the question to ask. More coupling bad, self-contained good :)

@itamaro
Copy link
Contributor Author

itamaro commented Nov 19, 2025

thanks @encukou
since the same assertion is triggered when importing submodules and modules within packages if they use inittab, addressing the issue for one use-case should also apply to the other!
note that the "modules in packages" use-case is not the same as "inittab-based packages". the name & initfunc in inittab will always be used to create a module or submodule, and the existence of a parent package is orthogonal.
I will look into adding dedicated test cases for both scenarios (and decouple from the test I hijacked), although I'm not quite sure how to set things up such that some package will be available at runtime for _testembed to use (without assuming something about the external environment, or having test_embed arrange things accordingly before invoking _testembed)

@brettcannon
Copy link
Member

Allowing submodules in inittab looks doable. (@brettcannon, do you have any thoughts on BuiltinImporter loading submodules?)

I think it's fine to add. I suspect it didn't have it just because the need never came up.

@itamaro
Copy link
Contributor Author

itamaro commented Nov 23, 2025

@encukou I added new dedicated tests

Copy link
Member

@encukou encukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thank you!

Co-authored-by: Petr Viktorin <encukou@gmail.com>
@encukou
Copy link
Member

encukou commented Nov 26, 2025

Looks good, thank you!

@encukou encukou merged commit 27f62eb into python:main Nov 26, 2025
48 checks passed
@itamaro itamaro deleted the gh-140011-dotted-import branch November 26, 2025 16:20
@itamaro itamaro added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Nov 26, 2025
@miss-islington-app
Copy link

Thanks @itamaro for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks @itamaro for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Sorry, @itamaro and @encukou, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 27f62eb711720c215f9798ae30728ee5a1b4d442 3.14

@miss-islington-app
Copy link

Sorry, @itamaro and @encukou, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 27f62eb711720c215f9798ae30728ee5a1b4d442 3.13

@itamaro itamaro restored the gh-140011-dotted-import branch November 26, 2025 18:22
itamaro added a commit to itamaro/cpython that referenced this pull request Nov 26, 2025
…bedded modules from packages (pythonGH-141605)

(cherry picked from commit 27f62eb)
@bedevere-app
Copy link

bedevere-app bot commented Nov 26, 2025

GH-141986 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Nov 26, 2025
itamaro added a commit to itamaro/cpython that referenced this pull request Nov 26, 2025
…bedded modules from packages (pythonGH-141605)

(cherry picked from commit 27f62eb)
@bedevere-app
Copy link

bedevere-app bot commented Nov 26, 2025

GH-141987 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 26, 2025
gpshead pushed a commit that referenced this pull request Nov 29, 2025
…mbedded modules from packages (GH-141605) (#141986)

gh-140011: Delete importdl assertion that prevents importing embedded modules from packages (GH-141605)

(cherry picked from commit 27f62eb)
gpshead pushed a commit that referenced this pull request Nov 29, 2025
…mbedded modules from packages (GH-141605) (#141987)

gh-140011: Delete importdl assertion that prevents importing embedded modules from packages (GH-141605)

(cherry picked from commit 27f62eb)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants