Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

file with many FF xx sequences grinds to a halt #128

Closed
richardlehane opened this issue Jun 19, 2019 · 4 comments
Closed

file with many FF xx sequences grinds to a halt #128

richardlehane opened this issue Jun 19, 2019 · 4 comments
Assignees
Labels
bug
Milestone

Comments

@richardlehane
Copy link
Owner

@richardlehane richardlehane commented Jun 19, 2019

A file with lots of FF xx sequences can generate so many hits against https://nationalarchives.gov.uk/PRONOM/fmt/134 that matching grinds to a seeming halt. Even a 500 byte file filled with FF xx sequences can take > 30 seconds to complete.

Can be "solved" (only as a work-around) by building a signature file without fmt/134: roy build -exclude fmt/134

Proper solution means optimising the matching code in the bytematcher

report and sample file provided by @fozboz

@richardlehane richardlehane added the bug label Jun 19, 2019
@richardlehane richardlehane added this to the 1.7.13 milestone Jun 19, 2019
@richardlehane richardlehane self-assigned this Jun 19, 2019
@richardlehane richardlehane modified the milestones: 1.7.13, 1.7.14 Aug 13, 2019
@jesswhyte

This comment has been minimized.

Copy link

@jesswhyte jesswhyte commented Oct 31, 2019

I am running another collection from the same donor and coming up against this issue again. Looking at some of the problem files (Sound Designer II Audio Files (.sd2)), they also have long FF sequences.

@richardlehane

This comment has been minimized.

Copy link
Owner Author

@richardlehane richardlehane commented Oct 31, 2019

Hi Jess, I've not forgotten this bug, it has proven pretty tricky to resolve, but have been working on it, unfortunately mostly in my head. Thank you for the extra incentive to fix, I hope to release something in the next few weeks!

@richardlehane

This comment has been minimized.

Copy link
Owner Author

@richardlehane richardlehane commented Jan 6, 2020

Just an update on this issue: I've finally got a working solution to this (on the "develop" branch") & it will be in the next sf release. I'll time the release to follow the next PRONOM update (expected next week - https://twitter.com/Britpunk80/status/1207331770301108229)

@richardlehane

This comment has been minimized.

Copy link
Owner Author

@richardlehane richardlehane commented Jan 23, 2020

I believe this is now fixed in 1.8.0. Please re-open if you encounter any similar issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.