New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RATIS-470. De-couple the LogService "index" from the RaftLog "index" #7

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
1 participant
@joshelser
Copy link
Member

joshelser commented Jan 9, 2019

We've been using the Raft log index as the "record offset" that we expose
to users via the LogService API. However, because one message that we push
to the Raft log may contain many LogService records, this generates
incorrect results.

We have to "hide" the Raft log index, and identify just the LogService records
so that we're exposing only their Log's information (e.g. none of the interal
Raft quorum configuration entries).

RATIS-470. De-couple the LogService "index" from the RaftLog "index"
We've been using the Raft log index as the "record offset" that we expose
to users via the LogService API. However, because one message that we push
to the Raft log may contain many LogService records, this generates
incorrect results.

We have to "hide" the Raft log index, and identify just the LogService records
so that we're exposing only their Log's information (e.g. none of the interal
Raft quorum configuration entries).
@joshelser

This comment has been minimized.

Copy link
Member

joshelser commented Jan 9, 2019

@chrajeshbabu @VladRodionov can you take a look for me? :)

@joshelser

This comment has been minimized.

Copy link
Member

joshelser commented Jan 9, 2019

Let me try to explain how this works now:

Take a "log" of data that looks like this:

["a", "b", "c", "d", "e"]

But, our Raft log may look like this:

[ config-entry, "a", ["b", "c", "d", "e"], config-entry]

The 3rd element in the above (["b", "c", "d", "e"]) would be created by the API call for LogWriter#write(List<ByteBuffer). I think it is the LogStateMachine's job to unwrap this "complicated" physical representation and make the last example "appear" to be the first via the LogService API.

Let me know if this doesn't make sense. I want to also capture this in docs somewhere..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment