Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: provide an temporary linearalizability mode (for taking consistent backups) #36208

Open
Bessonov opened this issue Mar 27, 2019 · 5 comments

Comments

@Bessonov
Copy link

commented Mar 27, 2019

Is your feature request related to a problem? Please describe.
This is a follow up from forum discussion.

Possibility to create data consistent backups is the most important feature of every database. Current implementation creates a consistent backup in regard to global state of CRDB. While the global state is OK from operational point of view, this is very bad for backups. There is no way to tell which data is not consistent.

High availability of CRDB doesn't help here, because HA isn't backup solution.

Describe the solution you'd like

As described in forum post, the main idea is to put CRDB temporary in linearizable mode. This can be achieved by temporal hold of writes (like google spanner) for probably 250ms or something global (like clock) to coordinate writes with reduced write throughput for same 250ms.

From my point of view, temporal delay of writes or reduced write throughput is totally OK, if there is a need for data consistent backups, even if it takes more than 250ms.

Describe alternatives you've considered
Alternatives are welcome!

Additional context
I don't know if there is an easy way to implement it in CRDB. But it is essential.

@roncrdb roncrdb added this to Triage in Bulk IO via automation Apr 2, 2019

@Bessonov

This comment has been minimized.

Copy link
Author

commented Apr 25, 2019

I have another idea that might be much easier to implement. What do you think about some sort of global transaction to retrieve a global time travel timestamp? Like (pseudo code):

BEGIN GLOBAL WRITE;

SELECT CURRENT SYSTEM TIME;

COMMIT;

Which lock all tables for writes, get consolidated timestamp and unlock all tables again. After that you can use returned value (ex. 2016-01-25T10:10:10.555555) to perform global consistent time travel:

cockroach dump <database> <flags> --as-of='2016-01-25T10:10:10.555555'

Beside of much easier implementation and consistent backups, you get two benefits:

  1. most small locking window to get consistent state timestamp for the time travel, not only for backups
  2. developers can use it to enforce linearizability if they put DML queries in the GLOBAL WRITE transaction

What do you think, guys?

@Bessonov

This comment has been minimized.

Copy link
Author

commented Jun 15, 2019

Hey @rolandcrosby @awoods187 , any news on this topic?

@Bessonov

This comment has been minimized.

Copy link
Author

commented Jul 6, 2019

A friendly reminder @awoods187 @rolandcrosby @roncrdb @justinj .

@rolandcrosby

This comment has been minimized.

Copy link
Collaborator

commented Jul 8, 2019

Hi @Bessonov, thank you for your patience, and sorry for not getting back on this earlier. We don't have any work planned on this for the 19.2 release this fall, but I've taken note of the problem you're trying to solve and am investigating possible solutions for future releases.

@rolandcrosby rolandcrosby changed the title Creation of data consistent backups Prevent causal reverse anomalies in backups via temporary linearalizability Jul 8, 2019

@rolandcrosby rolandcrosby removed this from Triage in Bulk IO Jul 8, 2019

@rolandcrosby rolandcrosby added this to Triage in Bulk I/O via automation Jul 8, 2019

@dt dt changed the title Prevent causal reverse anomalies in backups via temporary linearalizability core: provide an temporary linearalizability mode (for taking consistent backups) Jul 11, 2019

@dt dt added this to Incoming in Core via automation Jul 11, 2019

@dt dt removed this from Triage in Bulk I/O Jul 11, 2019

@Bessonov

This comment has been minimized.

Copy link
Author

commented Jul 16, 2019

@rolandcrosby thank you very much for your response! I would be very happy if it can get into 20.1 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.