Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unix-ffi: Isolate packages from python-stdlib version of os #672

Closed
wants to merge 2 commits into from

Conversation

jefferyto
Copy link

Currently, installing email.message (or any package that requires it, i.e. email.feedparser, email.parser, http.client) or glob causes the python-stdlib version of os to be installed instead of the unix-ffi version.

This adds unix-ffi versions of os-path and uu so that the unix-ffi version of os is installed in these cases.

jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 30, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
@jimmo
Copy link
Member

jimmo commented May 31, 2023

Thanks for the PR @jefferyto. Sorry the unix-ffi thing is a bit complicated, but I think this is a very reasonable way to solve this.

jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request May 31, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
@jefferyto
Copy link
Author

A more general solution could be to persist the unix_ffi=True flag into nested require() calls, that would allow a top-level unix-ffi package to include non-unix-ffi packages and have the stream of requires flow back into unix-ffi when necessary, but I'm not sure how difficult this would be or if there is any interest in this kind of functionality.

jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request Jun 2, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
jefferyto added a commit to jefferyto/openwrt-packages that referenced this pull request Jun 3, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 9390bd0)
AuthorReflex pushed a commit to AuthorReflex/packages that referenced this pull request Jun 7, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
UMB8998 pushed a commit to UMB8998/openwrt-packages that referenced this pull request Jun 20, 2023
The package has been reworked to install the same files that are
available to be downloaded/installed by mip, the package manager new to
MicroPython 1.20.0.

This also splits the original target package into four:

* micropython-lib
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains mpy bytecode files

* micropython-lib-src
  * Includes packages common to all MicroPython ports (python-stdlib,
    python-ecosys, micropython)
  * Contains py source files

* micropython-lib-unix
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains mpy bytecode files
  * Installs a specific launcher (micropython-unix) that adds the Unix
    package directory into MicroPython's library path

* micropython-lib-unix-src
  * Includes packages specific to the MicroPython Unix port (unix-ffi)
  * Contains py source files

This also updates the package license, title, and description.

Patches:

* 001-build-unix-ffi.patch

  This enables the repo build script to also build the Unix-specific
  packages. Not sure if upstream is open to accepting this since the
  Unix-specific packages don't fit well into the existing package
  distribution mechanism.

* 002-add-unix-ffi-os-path.patch and 003-add-unix-ffi-uu.patch

  These fix instances where the unix-ffi version of the os package is
  overridden by the python-stdlib version. These have been submitted to
  upstream: micropython/micropython-lib#672

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This package reuses the code from the python-stdlib version of `os-path`
but requires the unix-ffi version of `os`.

This also updates `glob` to require this version of `os-path`.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This package reuses the code from the python-stdlib version of `uu` but
requires the unix-ffi version of `os-path`.

This also updates `email.message` to require this version of `uu`.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
@dpgeorge
Copy link
Member

dpgeorge commented Feb 8, 2024

See micropython/micropython#13620 for an alternative solution.

@dpgeorge
Copy link
Member

dpgeorge commented Feb 8, 2024

A more general solution could be to persist the unix_ffi=True flag into nested require() calls

That's effectively what the above-linked PR does.

@dpgeorge
Copy link
Member

This was solved differently as per the two comments just above. See 23df50d

@dpgeorge dpgeorge closed this May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants