Add support for Python 3.4, 3.5 and 3.6 #168
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In Python3 dicts don't have an iteritems method, we use six.iteritems instead. In Python3 dict's values() and keys() methods don't return lists, we have to cast them explicitly.
six.string_types is basestring in Py2 and str in Py3
The package is called ConfigParser on Py2 and configparser on Py3
On py3 Popen.communicate returns bytes for standard streams. This commit turns on textmode (universal_newlines=True) to return a string on Py2 and Py3.
On Py3 this would return bytes otherwise.
Also updates the pip installs to upgrade packages if necessary.
Travis can't find a python 3.5 binary. This commit configures pyenv in the before_install section so that tox can find 3.5.
Tox "factors" provide better ways of configure conditional dependencies and commands for different tox environments, than named testenv sections. This commit merges the unittest and the pylint/bandit testenv sections back into one default testenv section and manages conditional deps/commands using factors. Additionally, the commit adds Python versions 3.4, 3.5 and 3.6
Modifies .travis.yml to run tox for each TOXENV with the corresponding Python version. Before, we started all tox builds with the same Python version and had tox/pyenv create the test virtualenv with the respective Python version, which threw pyenv errors.
Use max Python version supported by both bandit and pylint for
lint environment.
For the unittest environments py{VERSION}-unittest tox
automatically selects the desired VERSION.
The commit also adds a comment about how to run an individual
tox environment.
Pylint and bandit flag different things in different Python versions, hence it's a good idea to run them in every tox Python test environment. This commit removes the sophisticated testenv case handling with tox factors from previous commits and makes each Python test environment run first pylint, then badit and eventually the unittests.
In newer Python versions using log.warn makes pylint complain about deprecation.
Exclude version dependent imports from code coverage for coverage consistency across versions.
lukpueh
force-pushed
the
add-py3-support
branch
from
df255c0
to
0a4bff5
Compare
1 similar comment
SantiagoTorres
approved these changes
Jan 11, 2018
| @@ -146,7 +146,7 @@ def compute_keyid(pubkey_packet_data): | |||
| hasher.update(b'\x99') | |||
| hasher.update(struct.pack(">H", len(pubkey_packet_data))) | |||
| hasher.update(bytes(pubkey_packet_data)) | |||
| return binascii.hexlify(hasher.finalize()) | |||
| return binascii.hexlify(hasher.finalize()).decode("ascii") | |||
There was a problem hiding this comment.
I wonder if for all of these instances we want an errors='ignore' or some sort of error handling...
There was a problem hiding this comment.
I think just blowing up is better. What we could do is to catch exceptions and re-raise with a redacted error message. Maybe worth a note in #126.
|
|
||
| stdout_str = stdout_file.read() | ||
| stderr_str = stderr_file.read() | ||
| stdout_str, stderr_str = process.communicate() |
There was a problem hiding this comment.
I thought we had opted to not do this due to performance concerns/buffering. Is this still the case?
There was a problem hiding this comment.
I timed the function call for ls -Rl / and it executed without error in about 4 minutes, returning a stdout string of about 170MB.
>>> import timeit
>>> timeit.Timer("in_toto.runlib.execute_link(['ls', '-Rl', '/'], True)", setup="import in_toto.runlib").timeit(1)
247.98906087875366The timed ls -Rl in bash takes longer probably due to writing the ouput to the terminal.
$ time ls -Rl /
real 4m57.299s
user 0m51.453s
sys 2m20.026sI think it's fine for now. Also it is only a temporary solution, if we decide to adapt #160
|
Let me know if you think it is good to merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes issue #:
Closes #23
Description of the changes being introduced by the pull request:
Add support for Python 3.4, 3.5 and 3.6 (in addition to Python 2.7 support).
Commits whose titles start with
Py2/3contain all the changes for cross compatibility. The remaining commits update Travis and tox configs to run linters and unittests on all versions and fix coverage issues.Please verify and check that the pull request fulfills the following
requirements: