-
Notifications
You must be signed in to change notification settings - Fork 288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a new format which can support files longer than 2GB. #127
Conversation
Both the old format and the new format can be read by QueueFile. Opening an old file will automatically use the old format. New files automatically receive the new format unless a boolean is specified to force the legacy format.
4909c79
to
10be405
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me overall.
- Should QueueFile.toString include the version?
- We should add tests to verify v1 format can be read by v2.
- Add tests to verify format is updated when file is zeroed.
* | ||
* Header (32 bytes): | ||
* 1 bit Versioned indicator [0 = legacy (see "Legacy Header"), 1 = versioned] | ||
* 31 bits Version, always 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
31 bits seems excessive. I think 15 is sufficient and we could use a short instead. Though the overhead of adding code to deal with short just for this might not be worth it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes the header an even 32 bytes
Behavior and tests in a separate PR |
@@ -581,8 +673,8 @@ public void clear() throws IOException { | |||
|
|||
if (zero) { | |||
// Zero out data. | |||
raf.seek(HEADER_LENGTH); | |||
raf.write(ZEROES, 0, INITIAL_LENGTH - HEADER_LENGTH); | |||
raf.seek(headerLength); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it worth saying (on this method or in the class docs) that we don't ever upgrade from legacy to modern format?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That won't be true when we're done. If legacy format isn't forced queues will be upgraded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM.
Nice work, LGTM. |
Both the old format and the new format can be read by QueueFile. Opening an old file will automatically use the old format. New files automatically receive the new format unless a boolean is specified to force the legacy format.
Closes #47.