Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uniqueness violations during check version appear to be silently ignored #1371

Closed
alecgrieser opened this issue Aug 19, 2021 · 0 comments · Fixed by #1373
Closed

Uniqueness violations during check version appear to be silently ignored #1371

alecgrieser opened this issue Aug 19, 2021 · 0 comments · Fixed by #1373
Labels
bug Something isn't working

Comments

@alecgrieser
Copy link
Contributor

If a new unique index is added to a store and that index can be built in-line during checkVersion, it appears that this can result in the index being built and marked as readable even if there are uniqueness violations. I believe this to be caused by the way that index states are handled, as the new indexes are not marked as write-only before building them, but then there's an optimization in markIndexReadable that is intended to allow us to skip some work if someone marks an already readable index as readable:

byte[] indexKey = indexStateSubspace().pack(index.getName());
CompletableFuture<Boolean> future = tr.get(indexKey).thenCompose(previous -> {
if (previous != null) {
CompletableFuture<Optional<Range>> builtFuture = firstUnbuiltRange(index);
CompletableFuture<Optional<RecordIndexUniquenessViolation>> uniquenessFuture = scanUniquenessViolations(index, 1).first();
return CompletableFuture.allOf(builtFuture, uniquenessFuture).thenApply(vignore -> {
Optional<Range> firstUnbuilt = context.join(builtFuture);
Optional<RecordIndexUniquenessViolation> uniquenessViolation = context.join(uniquenessFuture);
if (firstUnbuilt.isPresent()) {
throw new IndexNotBuiltException("Attempted to make unbuilt index readable" , firstUnbuilt.get(),
LogMessageKeys.INDEX_NAME, index.getName(),
"unbuiltRangeBegin", ByteArrayUtil2.loggable(firstUnbuilt.get().begin),
"unbuiltRangeEnd", ByteArrayUtil2.loggable(firstUnbuilt.get().end),
subspaceProvider.logKey(), subspaceProvider.toString(context),
LogMessageKeys.SUBSPACE_KEY, index.getSubspaceKey());
} else if (uniquenessViolation.isPresent()) {
RecordIndexUniquenessViolation wrapped = new RecordIndexUniquenessViolation("Uniqueness violation when making index readable",
uniquenessViolation.get());
wrapped.addLogInfo(
LogMessageKeys.INDEX_NAME, index.getName(),
subspaceProvider.logKey(), subspaceProvider.toString(context));
throw wrapped;
} else {
updateIndexState(index.getName(), indexKey, IndexState.READABLE);
clearReadableIndexBuildData(tr, index);
return true;
}
});
} else {
return AsyncUtil.READY_FALSE;
}
}).whenComplete((b, t) -> endRecordStoreStateWrite()).thenApply(this::addRemoveReplacedIndexesCommitCheckIfChanged);

In the case of an index during checkVersion, the index won't have a previous state, so the logic to check if there are any uniqueness violations will be skipped, which means the index will be marked as readable even though it has uniqueness violations.

@alecgrieser alecgrieser added the bug Something isn't working label Aug 19, 2021
alecgrieser added a commit to alecgrieser/fdb-record-layer that referenced this issue Oct 13, 2021
…ppear to be silently ignored

This validates now that when a unique index is added with duplicate entries that the build does not succeed, but also that the store opening operation isn't failed. Instead, it just means that the index is placed in an unbuilt state that will need to be resolved later (like all other uniqueness violations). This behavior means that (1) new indexes won't be built which have violations of their own invariants and (2) adding a new unique index does not put the user into a state where they might accidentally make a store impossible to open.
alecgrieser added a commit that referenced this issue Oct 13, 2021
…silently ignored (#1373)

* Fixes #1371: Uniqueness violations during check version appear to be silently ignored

This validates now that when a unique index is added with duplicate entries that the build does not succeed, but also that the store opening operation isn't failed. Instead, it just means that the index is placed in an unbuilt state that will need to be resolved later (like all other uniqueness violations). This behavior means that (1) new indexes won't be built which have violations of their own invariants and (2) adding a new unique index does not put the user into a state where they might accidentally make a store impossible to open.

This adds a new package-private class, `IndexUniquenessCommitCheck`, which the `FDBRecordStore` will create when asked to add a new check. It associates the check with the given index so that the pertinent checks for each index can be collected and waited on when needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant