-
-
Notifications
You must be signed in to change notification settings - Fork 33.6k
gh-142349: Implement PEP 810 #142351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
pablogsal
wants to merge
118
commits into
python:main
Choose a base branch
from
LazyImportsCabal:lazy
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
gh-142349: Implement PEP 810 #142351
Changes from all commits
Commits
Show all changes
118 commits
Select commit
Hold shift + click to select a range
1c691ea
Lazy imports grammar / AST changes
DinoV 3b0d745
Add __lazy_import__, check sys.modules before import
DinoV 9eef03c
Export lazy_import in imp module
DinoV de281fd
Add lazy import filter
DinoV 41ab092
Add compatiblity mode
DinoV 058bc6e
Flow import func through to lazy imports object and __lazy_import__
DinoV 6d7c87a
Reify lazy objects when accessed via the module object
DinoV f992ee7
Import lazy.get
DinoV 6a91132
Implement better error
pablogsal 03a419a
Fix reference and global dict in spezializing LOAD_GLOBAL
pablogsal e0878be
Remove copyright
pablogsal f9880bf
More fixes
pablogsal f3f5795
Fix recursive lazy imports and error path in bytecodes.c
pablogsal 44a3e46
fix offset in addr2line
pablogsal 73de8d0
Add global flag
pablogsal 07a633f
Draft: force * imports to be eager
pablogsal 20b14d9
Syntax restrictions for lazy imports
pablogsal 164423b
Fix submodules crash
pablogsal 00e7800
Implement disabling imports in try/except and * imports, report error…
DinoV 781eedb
Add PyExc_ImportCycleError and raise it when a cycle is detected
DinoV 9be59ec
Publish lazy imported packages on parent
DinoV b179da2
Re-enable eagerly returning imported module
DinoV 9078f57
Fix __lazy_modules__
pablogsal f67310c
Add sys.set_lazy_imports_filter
pablogsal 39c33df
Simplify grammar
pablogsal c8c8838
Move __lazy_imports__ check into the interpreter
DinoV 7c49405
Don't allow __lazy_imports__ to work in try/except
DinoV 5d6026a
Make imports in with blocks syntax errors
pablogsal c0c0d80
Highlight lazy imports in the new REPL
johnslavik 5ff0dd2
Merge pull request #2 from bswck/lazy-import-pyrepl-highlight
pablogsal 214b254
Fix global membership in LOAD_NAME
pablogsal e5e9592
C was a mistake
pablogsal aa85f9d
dir() doesn't reify module
DinoV 2799a4d
Merge pull request #7 from LazyImportsCabal/lazy_dir
pablogsal 7d07ae1
Add test case for external usage
DinoV c63198c
Move eager check for from imports into import from
DinoV 46b3b75
Merge pull request #8 from LazyImportsCabal/lazy_test
DinoV c5efb20
Expose LazyImportType in types module
DinoV 0c246bc
__import__ is loaded at reification time
DinoV d9ad012
Fix leaks and LOAD_ATTR specialization
DinoV fe526b4
Merge pull request #9 from LazyImportsCabal/cleanup
pablogsal 06b9110
Update sys module to conform with the PEP, add matching C API
DinoV ee77665
Merge pull request #10 from LazyImportsCabal/sys_c_api
DinoV a3ddde4
Fix specialization of load global
DinoV cdec6a6
Merge pull request #12 from DinoV/lazy_mt
pablogsal c3b4807
Fix bug in specialization and make reification atomic
pablogsal 2e19765
Fix another race
pablogsal 2af21a1
Merge pull request #14 from LazyImportsCabal/fix
DinoV 34ea0a5
Make __dict__ not reify to match the PEP
DinoV 5166d39
Allow try/except in with block
DinoV a0a9184
Merge pull request #16 from LazyImportsCabal/lazy_globals
pablogsal 1f6518d
Grab builtins dict from module
DinoV 90246cc
Merge pull request #17 from DinoV/lazy_builtins
DinoV 824ac88
Fix eager imports in try/except in global mode
pablogsal c075f5b
Merge pull request #18 from LazyImportsCabal/bad_global
DinoV 59110fc
Don't reify on REPL completion
pablogsal aeda7ac
Add tests for __lazy_import__
DinoV 8d57aca
Merge remote-tracking branch 'origin/main' into HEAD
DinoV d8f95f7
Merge pull request #21 from LazyImportsCabal/lazy_rebase
DinoV b743eb0
Merge pull request #19 from DinoV/lazy_tests_dunder
pablogsal e6cb131
Implement more of PEP 810
pablogsal 0409481
Add more tests
pablogsal 617e9d1
Add more tests
pablogsal db151a5
Merge remote-tracking branch 'upstream/main' into lazy
pablogsal ea120fc
Regen stuff
pablogsal 973a4fa
Ah yes....windows
pablogsal 266fd4d
More windows stuff and news
pablogsal 441602a
Updated versions
pablogsal e6633ff
Update Doc/whatsnew/3.15.rst
pablogsal 2f642c8
Add support in IDLE colorizer
StanFromIreland ab07b14
Merge pull request #22 from StanFromIreland/lazy-idle
pablogsal 76846fe
Address feedback
pablogsal 0b36549
Fix tests
pablogsal 971d395
Update Doc/whatsnew/3.15.rst
pablogsal 0c019fd
Ruff fixes
pablogsal 8e1b20a
JIT fixes
pablogsal 70e5ce7
More jit fixes
pablogsal a39cd25
Fix smelly
pablogsal f70d4df
Fix smelly
pablogsal 510c200
Fix repl not coloring 'lazy' after 'from'
StanFromIreland d58b0cf
Doc fixes
pablogsal 8e4f292
brrrrrrrrrrrrrrrrrrrrrr
pablogsal 4cd4322
Merge pull request #24 from StanFromIreland/lazy-color
pablogsal 2d3681e
PeRfoRManCe
pablogsal 4cc6905
PeRfoRManCe
pablogsal 4f675fd
Amend return types for `PyImport_SetLazy*` functions
picnixz 4c3477b
moar fixes much refleaks wow
pablogsal 2f7d223
Merge pull request #25 from picnixz/patch-1
pablogsal 0985e2a
Doc updates
pablogsal 610af78
Address feedback
pablogsal 917b92e
More doc fixes
pablogsal e777866
Make error for invalid `lazy from __future__ ...` import prettier
StanFromIreland 952ac21
Regen limited ABI
pablogsal 33dc19e
Merge branch 'lazy' into lazy-future-error
StanFromIreland 80133a5
Fix error paths and edge cases
pablogsal b0adc30
Free threading fixes
pablogsal 470b9e4
TSAN fixes
pablogsal 41761e5
Update magic number
pablogsal 74b3efe
Merge pull request #26 from StanFromIreland/lazy-future-error
pablogsal 006ca9c
Regen frozenmain
pablogsal b5121b0
Fix WASI failure
pablogsal 093f08b
document return value for `PyImport_GetLazyImportsFilter`
picnixz 2a514d9
fix error path in `_PyImport_LoadLazyImportTstate`
picnixz 74f35c4
fix UAFs in `register_lazy_on_parent`
picnixz 3014816
fix error path in `_PyImport_LazyImportModuleLevelObject`
picnixz f96a99c
fix error path in `_imp__set_lazy_attributes_impl`
picnixz c4ed3c4
Various UAFs & cosmetic fixes in `Python/{bltinmodule,bytecode,ceval}…
picnixz 5000033
Fix missing `lazy`-related symbols for the JIT (#34)
picnixz a0a28c2
Merge branch 'main' into lazy
Yhg1s 023f806
Fix and improve `Objects/lazyimportobject.c` (#31)
picnixz 4b352dc
Fix doc merge snafu.
Yhg1s 7a0ddfd
Fix examples in the `ast` module docs (run by doctest).
Yhg1s 6fd8c59
Don't include the new C API functions
Yhg1s a05b50d
Add the new builtin types to the known-to-be-mutable-global lists in
Yhg1s ac80f2d
Fix some crashes
pablogsal 2d4bdcc
Document ImportCycleError and PyExc_ImportCycleError.
Yhg1s cd1878c
Use `self.assertIn/assertNotIn` instead of `self.assertTrue/assertFal…
StanFromIreland 31b7fe9
Fix more cases where lazy is not highlighted in repl (#36)
StanFromIreland File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -748,14 +748,15 @@ The :keyword:`!import` statement | |||||||||||
| pair: name; binding | ||||||||||||
| pair: keyword; from | ||||||||||||
| pair: keyword; as | ||||||||||||
| pair: keyword; lazy | ||||||||||||
| pair: exception; ImportError | ||||||||||||
| single: , (comma); import statement | ||||||||||||
|
|
||||||||||||
| .. productionlist:: python-grammar | ||||||||||||
| import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])* | ||||||||||||
| : | "from" `relative_module` "import" `identifier` ["as" `identifier`] | ||||||||||||
| import_stmt: ["lazy"] "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])* | ||||||||||||
| : | ["lazy"] "from" `relative_module` "import" `identifier` ["as" `identifier`] | ||||||||||||
| : ("," `identifier` ["as" `identifier`])* | ||||||||||||
| : | "from" `relative_module` "import" "(" `identifier` ["as" `identifier`] | ||||||||||||
| : | ["lazy"] "from" `relative_module` "import" "(" `identifier` ["as" `identifier`] | ||||||||||||
| : ("," `identifier` ["as" `identifier`])* [","] ")" | ||||||||||||
| : | "from" `relative_module` "import" "*" | ||||||||||||
| module: (`identifier` ".")* `identifier` | ||||||||||||
|
|
@@ -870,6 +871,56 @@ determine dynamically the modules to be loaded. | |||||||||||
|
|
||||||||||||
| .. audit-event:: import module,filename,sys.path,sys.meta_path,sys.path_hooks import | ||||||||||||
|
|
||||||||||||
|
|
||||||||||||
| .. _lazy-imports: | ||||||||||||
| .. _lazy: | ||||||||||||
|
|
||||||||||||
| Lazy imports | ||||||||||||
| ------------ | ||||||||||||
|
|
||||||||||||
| .. index:: | ||||||||||||
| pair: lazy; import | ||||||||||||
| single: lazy import | ||||||||||||
|
|
||||||||||||
| The :keyword:`lazy` keyword marks an import as lazy. It is a :ref:`soft keyword | ||||||||||||
| <soft-keywords>` that only has special meaning when it appears immediately | ||||||||||||
| before an :keyword:`import` or :keyword:`from` statement. | ||||||||||||
|
|
||||||||||||
| When an import statement is preceded by the :keyword:`lazy` keyword, | ||||||||||||
| the import becomes *lazy*: the module is not loaded immediately at the import | ||||||||||||
| statement. Instead, a lazy proxy object is created and bound to the name. The | ||||||||||||
| actual module is loaded on first use of that name. | ||||||||||||
|
|
||||||||||||
| Lazy imports are only permitted at module scope. Using ``lazy`` inside a | ||||||||||||
| function, class body, or :keyword:`try`/:keyword:`except`/:keyword:`finally` | ||||||||||||
| block raises a :exc:`SyntaxError`. Star imports cannot be lazy (``lazy from | ||||||||||||
| module import *`` is a syntax error), and :ref:`future statements <future>` | ||||||||||||
| cannot be lazy. | ||||||||||||
|
|
||||||||||||
| When using ``lazy from ... import``, each imported name is bound to a lazy | ||||||||||||
| proxy object. The first access to any of these names triggers loading of the | ||||||||||||
| entire module and resolves only that specific name to its actual value. Other | ||||||||||||
| names remain as lazy proxies until they are accessed. | ||||||||||||
|
|
||||||||||||
| Example:: | ||||||||||||
|
|
||||||||||||
| lazy import json | ||||||||||||
|
Comment on lines
+906
to
+907
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
For |
||||||||||||
|
|
||||||||||||
| print('json' in sys.modules) # False - module not loaded yet | ||||||||||||
|
|
||||||||||||
| # First use triggers loading | ||||||||||||
| result = json.dumps({"hello": "world"}) | ||||||||||||
|
|
||||||||||||
| print('json' in sys.modules) # True - now loaded | ||||||||||||
|
|
||||||||||||
| If an error occurs during module loading (such as :exc:`ImportError` or | ||||||||||||
| :exc:`SyntaxError`), it is raised at the point where the lazy import is first | ||||||||||||
| used, not at the import statement itself. | ||||||||||||
|
|
||||||||||||
| See :pep:`810` for the full specification of lazy imports. | ||||||||||||
|
|
||||||||||||
| .. versionadded:: next | ||||||||||||
|
|
||||||||||||
| .. _future: | ||||||||||||
|
|
||||||||||||
| Future statements | ||||||||||||
|
|
||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest you move this to the below section on lazy specifically.