Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Reverse timestamp issue with cell timestamps #2193
This is a request to document some limitations we came across with Bigtable, and a question about the use of
In our HBase table we do Put operations with a reversed timestamp, i.e.,
We do this to enforce a particular ordering, and this has worked fine with HBase.
When we used the
Will you make the
Also, how does this work while using
a) Say, I have an HBase table with cells with timestamp in milliseconds. And I export this to sequencefiles using HBase's
b) If I export the data in Bigtable using
c) How do these work if the HBase table had cells created with reverse timestamps, i.e.,
You are absolutely right that we need to document this behavior. The issue stems from the fact that bigtable stores timestamps as microseconds while hbase uses milliseconds. The hbase adapter tries to adjust the discrepancy by multiplying the hbase value by 1000. However the range of values that we can store is narrowed by this conversion. The hbase adapter did not consider reverse timestamps and assumed that if the timestamp higher than the highest storable value then we can just use the highest possible value. When reading it would convert it to just Long.MAX_VALUE.
The sequence file importer/exporter will use the hbase client under the hood. So any value higher than HBASE_EFFECTIVE_MAX_TIMESTAMP will be read back as Long.MAX_VALUE.
Other than documenting the behavior, there is no easy solution to this. As it stands
I'd like to understand your use case better this is the first time this has come up as an issue. Can you describe what the usecase for reverse timestamps in cell values?