Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add seq no powered optimistic locking support to the index and delete transport actions #36619

Merged
merged 5 commits into from Dec 15, 2018

Conversation

bleskes
Copy link
Contributor

@bleskes bleskes commented Dec 13, 2018

This PR add support for using sequence numbers to power optimistic concurrency control in the delete and index transport actions and requests. A follow up will come with adding sequence numbers to the update and get results.

Relates #36148
Relates #10708

@bleskes bleskes added >enhancement :Distributed/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. v7.0.0 v6.6.0 labels Dec 13, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

straight forward change. I like it. I don't like the name. left a suggestion since it's not expected :)

@@ -77,6 +78,8 @@
private static final ParseField RETRY_ON_CONFLICT = new ParseField("retry_on_conflict");
private static final ParseField PIPELINE = new ParseField("pipeline");
private static final ParseField SOURCE = new ParseField("_source");
private static final ParseField IF_SEQ_NO_MATCH = new ParseField("if_seq_no_match");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really? :D that name is very confusing. apparently expected_seq_no is not what you want do. bummer. I am definitely -1 on this one sorry. Same goes for any cas variants. I think these names are really not good. What about "replace" : { "seq_no" : x, "primary_term" : y}?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:) this is hard. The problem with json structure is that these need to go in a url parameter.

@@ -417,16 +427,16 @@ public BulkRequest add(BytesReference data, @Nullable String defaultIndex, @Null
if ("index".equals(action)) {
if (opType == null) {
internalAdd(new IndexRequest(index, type, id).routing(routing).version(version).versionType(versionType)
.setPipeline(pipeline)
.setPipeline(pipeline).ifMatch(ifSeqNoMatch, ifPrimaryTermMatch)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ifMatch no sorry -1 :) that's even worse than cas

@@ -215,6 +250,13 @@ public void readFrom(StreamInput in) throws IOException {
}
version = in.readLong();
versionType = VersionType.fromValue(in.readByte());
if (in.getVersion().onOrAfter(Version.V_7_0_0)) {
ifSeqNoMatch = in.readZLong();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can it be negative at this point? if not can we assert?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be -2 (unassigned). I wonder if we should call the validate method on these once deserialized and under assertion? I think that will give us a much better testing coverage?

Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left my suggestions, I will leave it to whoever makes the call on the naming. I will leave my reasoning here and that's it.

I think we use a CAS like approach internally. That approach expects a expected_primary_term and expected_seq_no. That's what I would name the rest params. the java API should be setExpected(long seqId, long primaryTerm) If it's not set you we don't expect anything. I tried to think of this as how would I write docs and explain it and from that perspecitve expected is the obvious solution. I personally feel the solutions with cas in the name or if are terrible and I would not want to go with. I understand that this is taste and if folks feel that is the way to go so be it.

@bleskes
Copy link
Contributor Author

bleskes commented Dec 14, 2018

run the gradle build tests 2

Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bleskes
Copy link
Contributor Author

bleskes commented Dec 15, 2018

Thanks @s1monw and @dnhatn . I will merge this in to keep momentum with the PR. I'll pick up the naming discussion first thing next week and will change the names to whatever the group decides.

@bleskes bleskes merged commit 733a6d3 into elastic:master Dec 15, 2018
@bleskes bleskes deleted the cas_seq_no_transport branch December 15, 2018 17:00
bleskes added a commit that referenced this pull request Dec 18, 2018
… transport actions (#36619)

This commit add support for using sequence numbers to power [optimistic concurrency control](http://en.wikipedia.org/wiki/Optimistic_concurrency_control)
in the delete and index transport actions and requests. A follow up will come with adding sequence
numbers to the update and get results.

Relates #36148
Relates #10708
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. >enhancement v6.6.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants