Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Support and test mutation log queries at intermediate timestamps #1380
I've removed all concepts of what "quantum" the storage layer has from tests, or anywhere else outside of the storage implementation itself. I think we've arrived at a place where the code in
In order to have consistent behavior for the HighWatermarks test across storage implementations, I've settled on returning a timestamp "just beyond" the primary key of the highest item. Since we're using
It is now the storage layer's job to round timestamp inputs up to whatever quantum the storage layer itself decides to use. This preserves all the necessary semantics and (very helpfully) we now have tests that verify this behavior.
@@ Coverage Diff @@ ## master #1380 +/- ## ========================================== + Coverage 65.54% 65.61% +0.06% ========================================== Files 52 52 Lines 3959 3961 +2 ========================================== + Hits 2595 2599 +4 + Misses 969 964 -5 - Partials 395 398 +3
Replace tests that wanted to predict the exact value of HighWatermark with tests that rely on the behavior of ReadLog to assert that the value returned is indeed correct. This allows different storage layers to use their own strategies for returning high watermarks. It also removes the requirement that HighWatermark values themselves be stored with nanosecond precision.
This also has the happy side effect of not imposing a particular time fidelity on the batch definition table. (Previous versions of this PR were going to force nanosecond fidelity)
* master: Define watermarks as micros (google#1384) Library for converting time.Time to sequencer watermarks (google#1381) Reduce log spam (google#1382) Support and test mutation log queries at intermediate timestamps (google#1380) In memory logs implementation (google#1375) Fix generic comparisons on protobuf messages (google#1379) Do pagination the right way (google#1378) Move the responsibility to pick an input log from storage to the frontend (google#1376) Init metrics for whole test file (google#1373) Break layering violation by using native types (google#1374) Switch Timestamp storage to mysql DATETIME (google#1369) Use testdb in integration tests (google#1371)