Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

py/builtinimport.c: Implement a "frozen overlay" using the filesystem. #8922

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jimmo
Copy link
Member

@jimmo jimmo commented Jul 18, 2022

Draft/WIP.

Haven't given much thought to code size (especially when the feature is disabled). The allow_frozen flag to the stat_ methods in particular needs some thought.

I considered a few other ways of implementing this, in particular making it detect the filesystem version of the file while searching sys.path but this creates a complicated interaction with things like loading the parent modules. Instead this approach says "a frozen file can be overridden by a filesystem file" rather than interfering with file discovery.

Filesystem means "VFS", so note this would work with mpremote mount too... (perhaps via #8914 we could have a way to make mpremote construct this "files that have changed relative to freezing" directory to mount)


This allows frozen files to be overridden by filesystem .py or .mpy files from a specific path specified in micropython.frozen_overlay.

The idea is that during development, a whole directory tree might be frozen
but for fast testing iteration it can be useful to just replace a couple
of files. This allows the same directory structure to be defined, but any
time a file is about to be loaded from frozen, the same path is attempted
in the overlay path.

Note: the overlay path should not be part of sys.path, otherwise it will
prevent the frozen path from being found.

This allows frozen files to be overridden by filesystem .py or .mpy files
from a specific path.

The idea is that during development, a whole directory tree might be frozen
but for fast testing iteration it can be useful to just replace a couple
of files. This allows the same directory structure to be defined, but any
time a file is about to be loaded from frozen, the same path is attempted
in the overlay path.

Note: the overlay path should not be part of sys.path, otherwise it will
prevent the frozen path from being found.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
@jimmo
Copy link
Member Author

jimmo commented Jul 18, 2022

To make it a bit more clear...

If the following tree is frozen:

a
  - __init__.py
  - b
    - c.py
    - d.py

but with micropython.frozen_overlay("overlay") and on the vfs:

overlay
  - a
    - b
      - __init__.py
      - c.py
      - e.py

Then the frozen files will all be loaded as normal, except for c.py which will be loaded from the filesystem. Note that a/b/__init__.py will not be loaded, and import a.b.e will fail because they do not exist in the frozen set.

Filesystem files can be .py or .mpy (and the usual rules that .py takes precedence over .mpy applies).

@jimmo
Copy link
Member Author

jimmo commented Aug 25, 2022

It was pointed out to me (thanks @tsukasa-au) that this has some resemblance to PEP420.

@projectgus
Copy link
Contributor

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants