[python3 support] Investigate what is needed

[sbellem]

Given the various specialized dependencies, especially the cryptographic ones, this issue is simply concerned with evaluating how feasible it would be to port HoneyBadgerBFT to Python 3 in a reasonable time frame.

The result of the evaluation will be documented here in this issue.

blockers

UPDATE: No more blockers. zfec now supports Python 3 -- PR tahoe-lafs/zfec#4 was merged.

zfec does not support Python 3 at the moment (related issue: tahoe-lafs/zfec#1)
There's a PR 😄 --> tahoe-lafs/zfec#4 !

Dependencies

lib	Latest Python 3 support	PyPI: release	PyPI: Py version	PyPI: Uploaded on
gevent	3.7	1.3a1	3.7	recently
gmpy2	3.6	2 2.1.0a1	3.6	recently
pysocks	3.6	1.6.8	3.6	recently
zfec	3.7	1.5.0	3.7	recently
Charm-Crypto	3.6 (from github)	0.43	3.4	2015-08-14
ecdsa	3.6 (from github)	0.13	3.4	2015-02-07
gipc	3.6 (from github)	0.6.0	3.4	2015-07-22
pycrypto	3.4	2.6.1	3.4	2014-06-20

Notes

Some details to watch out for.

charm-crypto

The latest charm-crypto cannot be installed simply with pip as there is an error when setting config values. This can be worked around by cloning the repo and performing the configuration to create a config file and running make, and then pip installing. So not a big deal, but inconvenient.

• Will look if an issue has already been filed, and if not will file one.

pycrypto

• Latest commit pycrypto/pycrypto@7acba5f on Jun 20, 2014
• 50 opened pull requests (https://github.com/dlitz/pycrypto/pulls)
• 111 opened issues (https://github.com/dlitz/pycrypto/issues)

• Figure out what's up with the project.

[amiller]

This reminds me that zfec will also be a dependency of honeybadger-go. Maybe the solution in either case will be to give zfec an interface so it can be called from a separate process?

[sbellem]

This reminds me that zfec will also be a dependency of honeybadger-go. Maybe the solution in either case will be to give zfec an interface so it can be called from a separate process?

I'm not sure what's the best approach for Go. I guess that ideally you would want a Go implementation of zfec, but time wise that is probably not ideal.

As for Python I just noticed that there was already a PR for Python 3 support (tahoe-lafs/zfec#4) and it is not so big. So I would prefer helping with getting this PR through, assuming that there's not too much work left to be done 😄 !

[sbellem]

UPDATE: Python 3 only port is under branch https://github.com/sbellem/HoneyBadgerBFT/tree/python3-port

All tests are passing, and the coverage is at 79%.

Things that are left to do (see Porting Python 2 Code to Python 3 for help):

• bring code coverage to 100% or nearby ([test:coverage] measure branch coverage #40, [test] add more unit tests for tpke module #48) -- work in PR Increase dev code coverage #58
• Run source code analyzers (e.g. pylint, flake8, etc) and fix errors ([test] add source code check to travis (pep8, etc) #33).
• clean up code changes
• see whether it makes sense to keep Python 2 support -- can be dropped :)
• add python 3 classifiers in setup.py
• ideally, run experiments/benchmarks with Python 3 version to make sure it still works ([dev] update/fix experiments #45) will be handled separately

[amiller]

This is fantastic work, thanks for being so systematic and looking into the zfec dependency especially. I appreciate that you set up the dependency link in the meantime to your own fork while also nudging them to merge the pull request!

I would be happy to adopt python3 even without maintaining python2 compatibility for this.

[sbellem]

Thanks to @tpltnt, pyutil now supports Python 3! This is a step forward for the zfec PR: tahoe-lafs/zfec#4 (comment).

[sbellem]

This is a step forward for the zfec PR: tahoe-lafs/zfec#4 (comment).

zfec now supports Python 3.

[sbellem]

Closing as it was handled in initc3/HoneyBadgerBFT-Python#3