New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Order of hashes no longer canonical #110

Merged
merged 1 commit into from Jan 31, 2019

Conversation

Projects
None yet
2 participants
@peterbe
Copy link
Owner

peterbe commented Jan 30, 2019

Fixes #105

CC @SomberNight

Here's how I tested it:

First observe that for requests the sha256 digests appear in this order:

  1. 7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b
  2. 502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e
▶ curl https://pypi.org/pypi/requests/json | jq '.releases["2.21.0"]' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 107k 100 107k 0 0 483k 0 --:--:-- --:--:-- --:--:-- 485k [ { "comment_text": "", "digests": { "md5": "ed3af234ffcad0b3c1e521e1dfde19be", "sha256": "7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b" }, "downloads": -1, "filename": "requests-2.21.0-py2.py3-none-any.whl", "has_sig": false, "md5_digest": "ed3af234ffcad0b3c1e521e1dfde19be", "packagetype": "bdist_wheel", "python_version": "py2.py3", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*", "size": 57987, "upload_time": "2018-12-10T15:40:08", "url": "https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl" }, { "comment_text": "", "digests": { "md5": "1bcd0e0977c3f8db1848ba0e2b7ab904", "sha256": "502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e" }, "downloads": -1, "filename": "requests-2.21.0.tar.gz", "has_sig": false, "md5_digest": "1bcd0e0977c3f8db1848ba0e2b7ab904", "packagetype": "sdist", "python_version": "source", "requires_python": ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*", "size": 111528, "upload_time": "2018-12-10T15:40:11", "url": "https://files.pythonhosted.org/packages/52/2c/514e4ac25da2b08ca5a464c50463682126385c4272c18193876e91f4bc38/requests-2.21.0.tar.gz" } ]

Run it the first time:

▶ python hashin.py -r /tmp/reqs.txt requests

▶ cat /tmp/reqs.txt
requests==2.21.0 \
    --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e \
    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b

Note that 502a824... comes before 7bf2a778... which is different from the JSON from Pypi.org.

Next, mess with the list of hashes manually:

▶ jed /tmp/reqs.txt

▶ cat /tmp/reqs.txt
requests==2.21.0 \
    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b \
    --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e

▶ python hashin.py -r /tmp/reqs.txt requests --dry-run

In other words, it didn't touch the file. It's because when it compares the previous hashes with the found hashes from pypi, it does it by comparing to sets.

Now really mess with it:

▶ cat /tmp/reqs.txt
requests==2.21.0 \
    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b \
    --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e

▶ jed /tmp/reqs.txt

▶ cat /tmp/reqs.txt
requests==2.21.0 \
    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b

▶ python hashin.py -r /tmp/reqs.txt requests

▶ cat /tmp/reqs.txt
requests==2.21.0 \
    --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e \
    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b

This time, the sets aren't equal so it goes to write down the hashes and when it does it does so in a sorted order.

@SomberNight

This comment has been minimized.

Copy link

SomberNight commented Jan 31, 2019

Looks OK, and seems to work for me too. Thanks!

@peterbe peterbe merged commit eed8a17 into master Jan 31, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@peterbe peterbe deleted the 105-order-of-hashes-no-longer-canonical branch Jan 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment