Update to Python 3.9.5

[hoodmane]

Closes #978

[rth]

Thanks for working on this @hoodmane !

[hoodmane]

A lot of packages are failing to build with the error:

[wasm-validator error in module] unexpected true: Imported global cannot be mutable,

This comes from wasm-opt. If I remove --mvp-features and add --enable-mutable-globals the error goes away. I guess I'll try patching emscripten with this...

[rth]

I think we should have this list either in that file or here, not both. Otherwise it's going to be annoying to maintain in sync.

[hoodmane]

I made it so that make_test_list.py writes this header into python_tests.txt. python_tests.txt is a weird mix of a generated file and a hand modified file...

[rth]

Were the issues at build time or at run time?

If I revert scikit-learn, scikit-image, statsmodels setup to the version on main, and then change the URL to a GitHub tag 3e28f4a they all build fine for me locally as part of this PR.

• scikit-image tests also pass
• scikit-learn and statsmodels fail to import which need more investigation.

[hoodmane]

At build time. I haven't tested whether they build locally until just recently, I was just looking at the CI. My modification to regenerate the statsmodels patches fixed the statsmodels build in CI. They fail with

error: no member named 'tp_print' in 'struct _typeobject'

in Cython-generated files. See here for example:

https://app.circleci.com/pipelines/github/hoodmane/pyodide/718/workflows/32426aa7-7c1c-4799-9d82-8b51f5bc152f/jobs/5937

[hoodmane]

If I revert scikit-learn, scikit-image, statsmodels setup to the version on main, and then change the URL to a GitHub tag 3e28f4a they all build fine for me locally as part of this PR.

Could you push this here?

[hoodmane]

On CI just now the Firefox statsmodels import test passes. Did you try to import it in Chrome or Firefox? We marked that test xfail in Chrome.

[rth]

Tests for statsmodels and scikit-image also passed when installed from GH with no other changes, but only after restarting CI. So I wonder if it means there is a function signature mismatch somewhere and it fails half the times.

[hoodmane]

there is a function signature mismatch somewhere and it fails half the times.

Is that a characteristic symptom of function signature mismatch? It does seem particularly flaky here.

[rth]

Well one of possible causes

pyodide/packages/scipy/meta.yaml

Lines 29 to 31 in f9a6899

	# linker and C flags are set to error on anything to do with function declarations being wrong. By default these
	# are just warnings, but in webassembbly, any conflicts mean that a randomly selected 50% of calls to the function
	# will fail. This is understandably bad. Better to fail at compile or link time

Actually maybe we should apply those flags globally to all packages.

[hoodmane]

Actually maybe we should apply those flags globally to all packages.

Sounds logical to me.

[hoodmane]

a randomly selected 50% of calls will fail

Is the use of the word "random" here accurate? Or does it really mean that some consistent fraction of the calls will fail? I don't really understand the mechanism that would cause the randomness here. Seems to me that there is a chance we are just hitting on typical CI flakiness here.

[hoodmane]

Okay all tests passed. I think we should proceed with merging this.

[rth]

Thanks for doing this work @hoodmane !

A few comments otherwise LGTM!

Please also,

• add a changelog entry
• don't forget to push the docker image and update .circleci/config.yml

[rth]

Is the use of the word "random" here accurate? Or does it really mean that some consistent fraction of the calls will fail? I don't really understand the mechanism that would cause the randomness here. Seems to me that there is a chance we are just hitting on typical CI flakiness here.

I think such issues might explain in part the CI flakiness. It does look like under some conditions emcripten compiled code fails non deterministically, there is some number of such issues in their issue tracker. Also apparently it's a documented behavior for va_arg,

If there is no next argument, or if type is not compatible with the type of the actual
next argument (as promoted according to the default argument promotions), random errors will occur. 

[hoodmane]

under some conditions emscripten compiled code fails non deterministically

This is certainly good to know. Sad though...

random errors will occur

Right... when I read these things I usually plug my ears and scream "unpredictable but repeatable". But one thing UB is allowed to do is fail non repeatably. In the va_arg example, presumably it depends what is in the memory on the stack next to the last argument, which will vary from run to run based on who-knows-what.