New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow files with more than 4GB #6

Open
wants to merge 7 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@J0s3f

J0s3f commented Aug 19, 2016

Please include this patch to allow large file patches

forthy42 added some commits Jun 14, 2016

comments, rename patch file location to patch file number to improve …
…understandability (no seeks in patch file)
@jjwhitney

This comment has been minimized.

Show comment
Hide comment
@jjwhitney

jjwhitney Aug 21, 2016

Owner

J0s3f,

There are some good changes in Bernd's patches, but also some that I can't merge.

  • Using 64-bit integers for file positions causes efficiency problems and should be done only for files > 4GB. (Generics might be the solution here?)
  • fseeko and ftello do not always use 64-bit integers, so we need to use fseeko64 and ftello64 instead.
  • Backward compatibility should be maintained for previous patches (for a while, anyway).
  • The ability to set granularity of diffs is important for finding changes in text files, so that cannot be removed (TOKEN_SIZE).
  • We shouldn't require the source files to be read into memory all at once.
  • The unmatched data in the patch should be kept separate so that compressible data is in contiguous blocks. I'm not sure what the motivation was for changing that?

I'll go through the changes to get fixes or improvements, but the merge as a whole is too aggressive.

Thanks!

Owner

jjwhitney commented Aug 21, 2016

J0s3f,

There are some good changes in Bernd's patches, but also some that I can't merge.

  • Using 64-bit integers for file positions causes efficiency problems and should be done only for files > 4GB. (Generics might be the solution here?)
  • fseeko and ftello do not always use 64-bit integers, so we need to use fseeko64 and ftello64 instead.
  • Backward compatibility should be maintained for previous patches (for a while, anyway).
  • The ability to set granularity of diffs is important for finding changes in text files, so that cannot be removed (TOKEN_SIZE).
  • We shouldn't require the source files to be read into memory all at once.
  • The unmatched data in the patch should be kept separate so that compressible data is in contiguous blocks. I'm not sure what the motivation was for changing that?

I'll go through the changes to get fixes or improvements, but the merge as a whole is too aggressive.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment