Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port upstream Pillow 10.0.0 #1

Merged
merged 2 commits into from
Oct 14, 2023
Merged

Conversation

andreittr
Copy link
Contributor

@andreittr andreittr commented Sep 11, 2023

This work is based off Pillow 10.0.0 configured on Linux x86_64 and adapted to work with Unikraft.
Files under importfix/ are correctly namespaced wrapper modules for any binary modules that we compile and link into Unikraft, and must be copied over to the python rootfs.
Selecting this library will add its files to the python rootfs build.

This depends on, or integrates with, the following PRs:

Makefile library inclusion order (x86, replace intrinsics with appropriate arm ones if on arm):

# Libs
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libcxx
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libcxxabi
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libunwind
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-intel-intrinsics
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-compiler-rt
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-musl
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-newlib
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libffi
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-lwip
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-bzip2
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-sqlite
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-openssl
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libuuid
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-zlib
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libdeflate
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libjpeg
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-openjpeg
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-giflib
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libpng
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libwebp
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-lzma
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-xz
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-zstd
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-libtiff
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-pixman
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-freetype
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-cairo
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-littlecms
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-fribidi
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-harfbuzz
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-raqm
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-geos
# Python stuff
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-python3
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-python-numpy
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-python-shapely
LIBS-y := $(LIBS-y):$(UK_LIBS)/lib-python-pillow

Signed-off-by: Andrei Tatar <andrei@unikraft.io>
This work is based off Pillow 10.0.0 configured on Linux x86_64 and
adapted to work with Unikraft.
Files under importfix/ are correctly namespaced wrapper modules for any
binary modules that we compile and link into Unikraft, and must be
copied over to the python rootfs.
Selecting this library will add its files to the python rootfs build.

Signed-off-by: Andrei Tatar <andrei@unikraft.io>
@razvand razvand requested a review from StefanJum October 1, 2023 04:26
@razvand razvand self-assigned this Oct 1, 2023
@razvand razvand added the enhancement New feature or request label Oct 1, 2023
@razvand razvand requested review from RaduNichita and mariasfiraiala and removed request for RaduNichita October 1, 2023 04:27
Copy link

@mariasfiraiala mariasfiraiala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @andreittr. Before starting testing, how would a Makefile with all the dependencies look like for pillow? I am mostly interested in the correct order of the libraries.

@andreittr
Copy link
Contributor Author

Thanks for this @andreittr. Before starting testing, how would a Makefile with all the dependencies look like for pillow? I am mostly interested in the correct order of the libraries.

Whoops, my bad, updated now. The order isn't strictly nailed down, it's just the one I was using during dev. You could likely mix them up as long as dependencies appear before the libs using them. I tried to be fairly paranoid about include paths not breaking.

@StefanJum
Copy link
Member

@andreittr any idea why I get this?

build/libpython_numpy/origin/numpy-1.25.0/numpy/core/src/common/npy_fpmath.h:19:6: error: #error No long double representation defined
   19 |     #error No long double representation defined

Copy link
Member

@StefanJum StefanJum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the comment, I get "No module named PIL__imaging" on testing. Again, might be just something I'm doing wrong on the fs setup.

Makefile.uk Show resolved Hide resolved
@andreittr
Copy link
Contributor Author

@andreittr any idea why I get this?

build/libpython_numpy/origin/numpy-1.25.0/numpy/core/src/common/npy_fpmath.h:19:6: error: #error No long double representation defined
   19 |     #error No long double representation defined

@StefanJum I believe you need this PR unikraft/lib-python-numpy#2 to solve those numpy include shenanigans; I had mentioned it among other fix PRs in unikraft/lib-python3#22 but forgot to add them to this PR as well, whoops.

Copy link
Member

@StefanJum StefanJum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good.
Reviewed-by: Stefan Jumarea stefanjumarea02@gmail.com

Copy link

@mariasfiraiala mariasfiraiala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great, thanks!

Reviewed-by: Maria Sfiraiala maria.sfiraiala@gmail.com

Copy link
Contributor

@razvand razvand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved-by: Razvan Deaconescu razvand@unikraft.io

@razvand razvand merged commit 131c8fa into unikraft:staging Oct 14, 2023
razvand pushed a commit that referenced this pull request Oct 14, 2023
Signed-off-by: Andrei Tatar <andrei@unikraft.io>
Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com>
Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com>
Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1
razvand pushed a commit that referenced this pull request Oct 14, 2023
This work is based off Pillow 10.0.0 configured on Linux x86_64 and
adapted to work with Unikraft.
Files under importfix/ are correctly namespaced wrapper modules for any
binary modules that we compile and link into Unikraft, and must be
copied over to the python rootfs.
Selecting this library will add its files to the python rootfs build.

Signed-off-by: Andrei Tatar <andrei@unikraft.io>
Reviewed-by: Stefan Jumarea <stefanjumarea02@gmail.com>
Reviewed-by: Maria Sfiraiala <maria.sfiraiala@gmail.com>
Approved-by: Razvan Deaconescu <razvand@unikraft.io>
GitHub-Closes: #1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants