Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numpy-1.26.0 #4213

Closed
slackline opened this issue Oct 11, 2023 · 6 comments
Closed

Numpy-1.26.0 #4213

slackline opened this issue Oct 11, 2023 · 6 comments
Labels
new package request

Comments

@slackline
Copy link

I've encountered a situation where the current numpy-1.25.2 which is available in Pyodide doesn't allow the dependencies of other packages to be resolved since they require numpy>=1.26.0 and numpy<2.0.0. I'd be very grateful if a version of numpy-1.26.0 could be made and provisioned for Pyodide, I'll have a go at working through the new packages instructions myself.

🐍 Package Request
This was referenced Oct 11, 2023
Open
Open
@slackline slackline mentioned this issue Oct 11, 2023
Closed
1 task
@slackline
Copy link
Author

Hi,

I've had a go at following the instructions at Creating a Pyodide Package to upgrade Numpy to 1.26.0. As the package already exists I opted to modify the packages/numpy.metadata.yaml file and bump the referenced version in the version: 1.26.0 field and then tried building the package in the Docker container.

For a reason I don't understand this resulted in numpy-1.25.2 being built (I haven't checked the wheel to see if the filename is consistent within the actual wheel).

neil@7381a232ff2b:/src$ ls -l dist/
total 18260
-rw-r--r-- 1 neil neil  185144 Oct 11 16:03 distutils-1.0.0.zip
-rw-r--r-- 1 neil neil  661086 Oct 11 16:03 hashlib-1.0.0.zip
-rw-r--r-- 1 neil neil   74054 Oct 11 16:03 lzma-1.0.0.zip
-rw-r--r-- 1 neil neil   50862 Oct 11 16:03 micropip-0.5.0-py3-none-any.whl
-rw-r--r-- 1 neil neil 3129973 Oct 11 16:03 numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl
-rw-r--r-- 1 neil neil 6973440 Oct 11 16:03 numpy-tests.tar
-rw-r--r-- 1 neil neil  795789 Oct 11 16:03 openssl-1.1.1n.zip
-rw-r--r-- 1 neil neil   48907 Oct 11 16:03 packaging-23.1-py3-none-any.whl
-rw-r--r-- 1 neil neil   48975 Oct 11 16:03 pydecimal-1.0.0.zip
-rw-r--r-- 1 neil neil  162279 Oct 11 16:03 pydoc_data-1.0.0.zip
-rw-r--r-- 1 neil neil    4332 Oct 11 16:03 pyodide-lock.json
-rw-r--r-- 1 neil neil  561947 Oct 11 16:03 sqlite3-1.0.0.zip
-rw-r--r-- 1 neil neil  851321 Oct 11 16:03 ssl-1.0.0.zip
-rw-r--r-- 1 neil neil 5117668 Oct 11 16:03 test-1.0.0.zip

I've tried checking /src/packages/numpy/build/log and it references 1.26.0 in many places, but also references 1.25.2 in others...

neil@7381a232ff2b:/src$ grep "1\.26" -n packages/numpy/build.log 
117:/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/system_info.py:2154: UserWarning: 
127:/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/system_info.py:2154: UserWarning: 
136:/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/system_info.py:2154: UserWarning: 
204:/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/system_info.py:1972: UserWarning: 
213:/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/system_info.py:1972: UserWarning: 
829:creating /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0
830:creating /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy
831:creating /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils
832:creating /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks
835:WARN: CCompilerOpt.dist_test[636] : CCompilerOpt._dist_test_spawn[770] : Command (/tmp/tmp6iehnh8l/cc -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g0 -fPIC -DPY_CALL_TRAMPOLINE -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/tmp/build-env-sg2w2_17/include -I/usr/local/include/python3.11 -Ibuild/src.emscripten_3_1_45_wasm32-3.11/numpy/core/src/common -Ibuild/src.emscripten_3_1_45_wasm32-3.11/numpy/core/src/npymath -c /src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.c -o /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.o -MMD -MF /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.o.d -march=native) failed with exit status 1 output -> 
837:emcc: error: '/src/emsdk/emsdk/upstream/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -DEMSCRIPTEN -Werror=implicit-function-declaration --sysroot=/src/emsdk/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/fakesdl -Xclang -iwithsysroot/include/compat -DNDEBUG -g3 -fwrapv -O3 -Wall -O2 -g0 -fPIC -DPY_CALL_TRAMPOLINE -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/tmp/build-env-sg2w2_17/include -Ibuild/src.emscripten_3_1_45_wasm32-3.11/numpy/core/src/common -Ibuild/src.emscripten_3_1_45_wasm32-3.11/numpy/core/src/npymath -c -MMD -MF /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.o.d -march=native -Werror=implicit-function-declaration -Werror=mismatched-parameter-types -Werror=return-type -Wno-return-type -O2 -g0 -fPIC -I/src/cpython/installs/python-3.11.3/include/python3.11 -I/src/cpython/installs/python-3.11.3/include/python3.11 /src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.c -o /tmp/tmp0ko4ucjd/src/packages/numpy/build/numpy-1.26.0/numpy/distutils/checks/test_flags.o' failed (returned 1)
2455:creating '/src/packages/numpy/build/numpy-1.26.0/dist/tmp07ej57tj/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl' and adding 'build/bdist.emscripten_3_1_45_wasm32/wheel' to it
3353:INFO: CCompilerOpt.cache_flush[863] : write cache to path -> /src/packages/numpy/build/numpy-1.26.0/build/temp.emscripten_3_1_45_wasm32-3.11/ccompiler_opt_cache_ext.py
3370:INFO: CCompilerOpt.cache_flush[863] : write cache to path -> /src/packages/numpy/build/numpy-1.26.0/build/temp.emscripten_3_1_45_wasm32-3.11/ccompiler_opt_cache_clib.py
3371:Successfully built /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl
3372:Unpacking wheel to /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl                                                               
3385:Repacking wheel as /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl...OK
neil@7381a232ff2b:/src$ grep "1\.25" -n packages/numpy/build.log 
806:set build/lib.emscripten_3_1_45_wasm32-3.11/numpy/_version.py to '1.25.2'
2452:Copying numpy.egg-info to build/bdist.emscripten_3_1_45_wasm32/wheel/numpy-1.25.2-py3.11.egg-info
2454:creating build/bdist.emscripten_3_1_45_wasm32/wheel/numpy-1.25.2.dist-info/WHEEL
2455:creating '/src/packages/numpy/build/numpy-1.26.0/dist/tmp07ej57tj/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl' and adding 'build/bdist.emscripten_3_1_45_wasm32/wheel' to it
3330:adding 'numpy-1.25.2.dist-info/LICENSE.txt'
3331:adding 'numpy-1.25.2.dist-info/METADATA'
3332:adding 'numpy-1.25.2.dist-info/WHEEL'
3333:adding 'numpy-1.25.2.dist-info/entry_points.txt'
3334:adding 'numpy-1.25.2.dist-info/top_level.txt'
3335:adding 'numpy-1.25.2.dist-info/RECORD'
3371:Successfully built /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl
3372:Unpacking wheel to /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl                                                               
3373:Unpacking to: /tmp/tmpuskxhaj6/numpy-1.25.2...OK
3374:Collecting numpy==1.25.2
3375:  Using cached numpy-1.25.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
3377:Successfully installed numpy-1.25.2
3385:Repacking wheel as /src/packages/numpy/build/numpy-1.26.0/dist/numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl...OK

I'm a little confused by the last line of each grep where it suggests that the build of numpy-1.26.0 has been packaged as numpy-1.25.2-cp311-cp311-emscripten_3_1_45_wasm32.whl.

Is there another way to go about upgrading packages and checking they build locally?

Perhaps more helpfully I've put in a PR to bump Numpy #4217

@jobovy
Copy link
Contributor

jobovy commented Oct 11, 2023

You have to also update the url field in meta.yaml. Note that you can upgrade using

pyodide skeleton pypi numpy --update

which would automatically update the url field as well.

@slackline
Copy link
Author

Ahha, thank you 👍

I've updated that field in the PR based on the URL for the source on PyPI.

Cheers @jobovy I've learnt loads today and that was the final piece in the jigsaw (maybe!).

@slackline
Copy link
Author

I have to update the sha256 field too, stupid of me to miss that 🤦

@jobovy
Copy link
Contributor

jobovy commented Oct 11, 2023

I have to update the sha256 field too, stupid of me to miss that 🤦

Sorry, my bad for not mentioning that. pyodide skeleton pypi numpy --update would have taken care of that as well.

@slackline
Copy link
Author

Don't worry, its my fault for not taking your advice nor reading the YAML in detail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new package request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Bumping numpy 1.25.2 > 1.26.1 slackline/pyodide
3 participants
@slackline @jobovy @ryanking13