Skip to content

Conversation

@wenkokke
Copy link
Collaborator

@wenkokke wenkokke commented May 6, 2025

This PR reorders the fields in TableConfig to ensure that the MergePolicy and MergeSchedule are adjacent.

(cfs :: MergeSchedule))
= TableConfig cmp csz cwba cbfa cfit dcp cfs
override confDiskCachePolicy' TableConfig {..}
= TableConfig {confDiskCachePolicy = confDiskCachePolicy', ..}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes with a risk of forgetting to override all relevant places when new fields are added. Not super likely that the policy occurs more than once in the config, but maybe still a good habit to a be more explicit about these things. For previous discussion and an alternative approach that is not quite as verbose: #641 (comment))

Comment on lines 299 to 300
confMergePolicy <- decodeVersioned v
confSizeRatio <- decodeVersioned v
confWriteBufferAlloc <- decodeVersioned v
confBloomFilterAlloc <- decodeVersioned v
confFencePointerIndex <- decodeVersioned v
confDiskCachePolicy <- decodeVersioned v
confMergeSchedule <- decodeVersioned v
pure TableConfig {..}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be okay to make a change to the serialisation format now, before the library gets released.

But I kind of like that you make the order explicit here, instead of it implicitly being defined in the datatype declaration (which had a risk of changing it without being aware of the consequences).

Comment on lines 286 to 294
encode TableConfig {..} =
encodeListLen 7
<> encode mergePolicy
<> encode sizeRatio
<> encode writeBufferAlloc
<> encode bloomFilterAlloc
<> encode fencePointerIndex
<> encode diskCachePolicy
<> encode mergeSchedule
where
TableConfig
mergePolicy
sizeRatio
writeBufferAlloc
bloomFilterAlloc
fencePointerIndex
diskCachePolicy
mergeSchedule
= config
<> encode confMergePolicy
<> encode confSizeRatio
<> encode confWriteBufferAlloc
<> encode confBloomFilterAlloc
<> encode confFencePointerIndex
<> encode confDiskCachePolicy
<> encode confMergeSchedule
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The risk here is that one might forget to encode a new field. However, the decodeVersioned implementation would fail to compile, hopefully making it clear that encode also needs to be updated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point here is that if you forget to encode a new field, the decode test fails, which hopefully lets you know that you messed up the encode function. Moreover, these functions are much less brittle depending on the order in the datatype. I've elaborated the pattern match to be explicit about names and total, which is much more verbose, but combines the best of both.

@wenkokke wenkokke force-pushed the wenkokke/move-conf-merge-schedule branch 4 times, most recently from ee3fdf3 to 0c41a34 Compare May 7, 2025 12:22
Copy link
Collaborator

@mheinzel mheinzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment, but this seems good to go either way.

confWriteBufferAlloc = confWriteBufferAlloc,
confBloomFilterAlloc = confBloomFilterAlloc,
confFencePointerIndex = confFencePointerIndex,
confDiskCachePolicy = confDiskCachePolicy'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changed and unchanged lines seem super similar, which makes it a bit hard to see where the update happens. Two ideas for making the updated field stands out more:

  • use a different name for the policy argument
  • use NamedFieldPuns in the other lines, so they're just confMergePolicy, etc., making it super clear that the field is untouched

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used the NamedFieldPuns suggestion.

@wenkokke wenkokke force-pushed the wenkokke/move-conf-merge-schedule branch 3 times, most recently from 375cb74 to 0ce2354 Compare May 8, 2025 14:11
@wenkokke wenkokke force-pushed the wenkokke/move-conf-merge-schedule branch from 0ce2354 to 75d28a7 Compare May 8, 2025 15:11
@wenkokke wenkokke enabled auto-merge May 8, 2025 15:56
@wenkokke wenkokke added this pull request to the merge queue May 8, 2025
Merged via the queue into main with commit 84c91d0 May 8, 2025
28 checks passed
@wenkokke wenkokke deleted the wenkokke/move-conf-merge-schedule branch May 8, 2025 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants