test and build wheels for Py3.{7,8,9,10} #3298
Conversation
Codecov Report
@@ Coverage Diff @@
## develop #3298 +/- ##
========================================
Coverage 79.53% 79.53%
========================================
Files 68 68
Lines 11781 11781
========================================
Hits 9370 9370
Misses 2411 2411 Continue to review full report at Codecov.
|
test wheels in a separate step that gets skipped for Py3.10. this is a work-around for a segfault that I cannot reproduce locally.
|
Oof, finally got everything to build for Python 3.10. There are some catches:
Please let me know what you think. |
|
The simpler the better :) You know way more about this subsystem than I do @mpenkov, so if you say it works (~works better), let's merge. |
|
OK, I re-enabled the tests and increased the tolerance for the test that was failing. It works, so I'm merging this now. |
| env: | ||
| PKG_NAME: gensim | ||
| REPO_DIR: gensim | ||
| BUILD_COMMIT: HEAD | ||
| PLAT: x86_64 | ||
| UNICODE_WIDTH: 32 | ||
| MB_PYTHON_VERSION: ${{ matrix.python-version }} # MB_PYTHON_VERSION is needed by Multibuild | ||
| TEST_DEPENDS: Morfessor==2.0.2a4 python-levenshtein==0.12.0 visdom==0.1.8.9 pytest pytest-cov mock cython nmslib pyemd testfixtures scikit-learn pyemd | ||
| TEST_DEPENDS: pytest mock testfixtures |
There was a problem hiding this comment.
Will removing pyemd (& perhaps others, as well) mean that certain tests we want run are skipped because of missing-libraries?
There was a problem hiding this comment.
It's possible, but this is for the purpose of testing the wheel, so running tests that require those dependencies isn't strictly necessary.
| import urllib.request | ||
| with tempfile.NamedTemporaryFile(suffix='.py') as fout: | ||
| urllib.request.urlretrieve("https://bootstrap.pypa.io/get-pip.py", fout.name) | ||
| subprocess.call([sys.executable, fout.name]) |
There was a problem hiding this comment.
From a security-in-depth perspective, fetching & immediately running unversioned raw code off the web spooks me, & I'd avoid it if at all possible. Sure, this appears to be from the right pypa.io domain, & thus reflecting the software of a team/project (Python Packaging Authority) which we're already implicitly trusting via things like pip/PyPI installs that the same people/processes control.
But I know the PyPI repository is both a vulnerability point, & a major focus of attackers, and thus also a place where (I hope & expect) code-provenance & system-security matters are taken very seriously. Maybe pypa.io is as rigorous – they should be! – but the naked https://bootstrap.pypa.io/ directory isn't quite offering the signals-of-rigor I'd most like to see.
Maybe the added risk is considered acceptable, maybe we have to use this as a hacky workaround for a while – but we should try to retire it the moment it's not necessary, and we could take extra steps to ensure we're getting a consistent version of get-pip.py. For example: we could grab it from the github project by version-hash at https://github.com/pypa/get-pip/blob/main/public/get-pip.py, to be sure we only get a new one when we choose to roll forward – so our builds always use the exact version we verified OK during testing of this PR, nothing later (either improved or compromised). If this in fact misses a useful true update, causing something to break someday – maybe that's OK, because that's enough time passed to double-check if this kludge is still required.
|
Sorry, I thought my 2 code review comments had been posted days ago – they were visible to me, even outside the 'changed files' page – but the |
No description provided.