Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transaction Size does not take into account read and write conflict ranges #87

Open
KrzysFR opened this issue May 12, 2018 · 1 comment
Labels
api Issues or changes related to the client API

Comments

@KrzysFR
Copy link
Member

KrzysFR commented May 12, 2018

Looking at the code in fdbclient (Transaction::commitMutations()) it looks like starting from API version 300+, the total transaction size is the sum of the mutated key/value (writes), as well as the size of all the read and write conflict ranges.

A write with key "FOO" and value '"Hello World"` should add the following overhead:

  • Mutations: (Set, 'FOO', 'Hello World').expectedSize() = 3 + 11 = 14 bytes
  • Read conflicts: None
  • Write conflicts: ('FOO', 'FOO\0') = 3 + 4 = 7 bytes

The total footprint of this operation would be 21 bytes instead of 14 bytes as is currently guesstimated by the .NET binding. This means that algorithms that try to batch data by following the transaction size are underestimating the actual size, and could fail with transaction_too_large while thinking this is ok.

@KrzysFR
Copy link
Member Author

KrzysFR commented Nov 27, 2019

There is a new API in fdb 6.2 that allows to get a estimate of the transaction size, which should be more precise than the current heuristic.

Only issue is that the new API is async, which is a breaking change, and also may change the perf characteristics of existing implementations.

@KrzysFR KrzysFR added the api Issues or changes related to the client API label Nov 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Issues or changes related to the client API
Projects
None yet
Development

No branches or pull requests

1 participant