Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Choose faster hash than MD5 #18

Closed
phseiff opened this issue Oct 23, 2020 · 4 comments
Closed

Choose faster hash than MD5 #18

phseiff opened this issue Oct 23, 2020 · 4 comments
Labels
fixed The issue was fixed. performance gh-md-to-html being too slow or too RAM-intensive

Comments

@phseiff
Copy link
Owner

phseiff commented Oct 23, 2020

gh-md-to-html currently uses MD5 to ensure no two identical images are saved in different locations, which is pretty slow. A faster hash algorithm might work wonders, especially since all we need is collision security and nothing more.
@phseiff phseiff added the performance gh-md-to-html being too slow or too RAM-intensive label Oct 23, 2020
@phseiff
Copy link
Owner Author

phseiff commented Oct 25, 2020 •
edited

gh-md-to-html now uses pythons builtin-hash-function instead of hashlib.md5 if there are less than 1000 images to hash, which speeds things up a little. The speed of the conversion of a bunch of files files I used to test went down from 12 minutes to 5 minutes.

@phseiff
Copy link
Owner Author

phseiff commented Oct 25, 2020

Additionally, the strings generated from the images that are then hashed are now much shorter, which further improves the time it takes to hash them.

@phseiff
Copy link
Owner Author

phseiff commented Oct 25, 2020

This should do for now, though I could think about storing the hashes to be reusable in case multiple MD files are converted in the same directory.

@phseiff phseiff added the fixed The issue was fixed. label Oct 25, 2020
@phseiff
Copy link
Owner Author

phseiff commented Apr 8, 2021

This has been open for quite some time now, and no one has complained so far, so I will go ahead and close this now.
Feel free to tell me if you are experiencing performance problems with the currently used hash.

@phseiff phseiff closed this as completed Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fixed The issue was fixed. performance gh-md-to-html being too slow or too RAM-intensive
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@phseiff