-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move to Python 3: dependencies status #2891
Comments
PyHive has no trove classifiers in its PyPI record... https://pypi.org/project/PyHive But it is being tested under all current versions of CPython https://github.com/dropbox/PyHive/blob/master/.travis.yml#L8 |
Same with pytest-watch... https://pypi.org/project/pytest-watch |
This is super exciting, didn't realize how close we'd be in terms of dependencies. For flask-sslify I planned to work on mozilla#562 and port Redash to use flask-talisman which also supports enforcing encrypted connections (on top of plenty of other security related things, which is what interests us at Mozilla). A bit like a kill two birds with one stone situation. Any general objections about this? |
@jezdez sounds good to me. 👍 |
I saw many ImportError when trying to run on python3. |
@arikfr Oh btw, we merged flask-talisman! Which means there shouldn't be any blocker to this? |
@jezdez We should probably run But there are probably other things that are blocking switching over to Python 3 in our code or dependencies. I'm just not sure what they are. 😳 |
|
The |
I think it counts python-geoip-geolite2 and python-geoip individually:
|
I've opened ibmdb/python-ibmdb#395 to fix caniusepython3 missing this. |
pydruid is also tested with Python 3.6: https://github.com/druid-io/pydruid/blob/master/.travis.yml |
Based on this, it looks like the dependencies are probably a non issue. Now there is only the small issue of figuring out if our code works as intended on Python 3 😅 |
My sense is that you should try putting apt install python3 python3-pip into your CirceCI builds and see where the tests fail. I think you are really close and it would be good to understand what is failing. |
Opened druid-io/pydruid#164 |
Yeah, running 2to3 (or some wrappers mentioned on https://docs.python.org/3/howto/pyporting.html) make sense as well. There is a pretty good high-level overview in The Conservative Python 3 Porting Guide which I think applies to Redash well. |
I would avoid 2to3 because it tends to break Py2 compatibility to get to Py3 compatibility. I would recommend using either futurize or modernize instead. These thin wrappers on top of 2to3 that preserve compatibility with bothPy2 and Py3 |
I'm not sure we will support Python 2 once we move to 3. |
Your call. However best practice is to run them both in a single codebase for at least a little time because the A/B testing can be invaluable. See the conservative guild and python docs on porting. |
A few Python 3 issues to watch out for in the porting... flake8 testing of https://github.com/getredash/redash on Python 3.7.1 $ flake8 . --count --select=E9,F63,F72,F82 --show-source --statistics
|
I setup a new CircleCI job to run tests on Python 3. It uncovered that our MySQL lib does not support Python 3: https://circleci.com/gh/getredash/redash/21775 I updated to https://circleci.com/gh/getredash/redash/21782 -- Update: new build fails too. I will iterate on the dependencies locally. |
See the comments at the bottom of mitsuhiko/python-geoip#5 |
Python 2 support after Python 3: I guess it depends on how much effort it will require. If possible we will keep support for both. Flake 8 tests: I assumed that the CircleCI job for Flake8 tests on Python 3 will fail if it doesn't pass 😳 Oops. |
There are a few more dependencies that fail to build with the Python 3 image. I noticed Update: it has a verbose output that should list them -- will check. |
Regarding whether to go 2/3 or straight to 3, I don't think it matters which strategy to use as long as we can run on Python 3 before the end of the year -- so chose the strategy that works best for you. FWIW I know a few distributors will support Python 2 going forward, but it'd cost us, which I don't think is a viable option either for Redash SaaS or the OSS users. TBH I fully expect to see Python 2 vulnerabilities to show up after the EOL. Handy tool: https://pythonclock.org/ |
This is not related to the issue, but I assume that the issue author or followers might have SAML enabled for their deployment and should be aware of the following Security Advisory: #5961. This affects all Redash versions and should be patched immediately. |
Thanks to the work of @cclauss our code is (more) ready for Python 3. I also ran caniusepython3 on our requirements files and the results are very encouraging:
python-geoip
has a Python 3 compatible alternative (python-geoip-python3
)flask-sslify
is marked as Python 3 compatible, but no new version was released for a long time.chromelogger
is unknown but we can either drop it or update to support Python 3 (~140 lines of code).restrictedpython
has an unreleased version 4 that supports Python 3.I'm not sure why
pyhive
is marked as not supported, because it is. Maybe we just need to update the version.I'm not sure about the state of
pytest-watch
, but we can always remove it.The text was updated successfully, but these errors were encountered: