CBG-4778: LWW conflict resolution for ISGR

[gregns1]

CBG-4778

• Adds HLV conflict resolution to active replicator. We need two defined given when we build active replicator we don't know what sub-protocol we're running in but also we need reference to pre 4.0 conflict resolution for pre upgraded docs.
• Adds lower level test around local wins and remote wins
• Adds higher level tests for conflict resolution around local wins and remote wins cases
• Needed to have the ability to mock the HLV on a doc to look a certain way for test cases around MV so added test helper AlterHLVForTest
• PR will not reconcile rev tree's at this time, later ticket for this
• PR takes into account work for CBG-4789

Pre-review checklist

• Removed debug logging (fmt.Print, log.Print, ...)
• Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
• Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Dependencies (if applicable)

• Link upstream PRs
• Update Go module dependencies when merged

Integration Tests

• GSI=true,xattrs=true https://jenkins.sgwdev.com/job/SyncGatewayIntegration/53/

[Copilot]

Pull Request Overview

This PR adds HLV (Hybrid Logical Vector) conflict resolution support for inter-sync gateway replication (ISGR). It implements both local wins and remote wins conflict resolution strategies for version vector-based conflicts, along with comprehensive test coverage for various conflict scenarios including merge version handling and attachment preservation.

• Adds HLV conflict resolution functions and infrastructure to support vector-based conflict resolution
• Implements comprehensive test suite covering local wins, remote wins, merge version scenarios, and attachment handling
• Updates existing tests to support both legacy and HLV conflict resolution modes

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
rest/replicatortest/replicator_test.go	Removes skips from tombstone tests and adds protocol version configuration
rest/replicatortest/replicator_conflict_test.go	Adds comprehensive HLV conflict resolution test suite with 1100+ lines of tests
rest/blip_api_crud_test.go	Removes skip from stat test
db/utilities_hlv_testing.go	Adds test helper function to alter document HLV for testing scenarios
db/util_testing.go	Updates function call to include conflict resolver parameter
db/sg_replicate_conflict_resolver.go	Adds HLV-aware conflict resolution functions and ResolveForHLV method
db/sg_replicate_cfg.go	Updates replicator config to support both legacy and HLV conflict resolvers
db/import_test.go	Updates function call to include conflict resolver parameter
db/hybrid_logical_vector_test.go	Adds unit tests for HLV conflict resolution algorithms
db/hybrid_logical_vector.go	Implements core HLV conflict resolution logic
db/crud_test.go	Updates function calls to include conflict resolver parameter
db/crud.go	Integrates HLV conflict resolution into document update flow
db/blip_handler.go	Updates BLIP handler to pass conflict resolver to HLV operations
db/active_replicator_config.go	Adds HLV conflict resolver function configuration

[Copilot]

Typo in comment: "syc" should be "sync"

[gregns1]

This is fixed?

[torcolvin]

I plan to review this more, but I've created some helper functions #7704 to deal with updating HLV. These duplicate/obviate some of the code here. I needed to create these standalone functions to write unit tests that couchbase lite can use to ensure matching behavior.

Those functions will be used for blip client testing and for ISGR.

[torcolvin]

There seem to be a few differences between resolveLocalWinsHLV and resolveDocLocalWins, some of which seem right, and some of which seem not right.

1. There's not a new revtree id that is created in this function. You would need a local revtree id to support 3.x CBL replication even after 4.x ISGR replication.
2. There's handling of branching tombstones. I'm not sure there will ever be branching since the local revtree is already overwritten by remote revtree by the time we are here? but I'm actually not sure.
3. tombstoneActiveRevision means that the old revision is added to the doc.History for revtree ids.

[gregns1]

1. New RevID is generated after this function is done inside PutExistsingCurrentVersion
   2 & 3. There is work after this to reconcile revTree's on conflict resolution, the idea here is work for this will take place there

[torcolvin]

Are there going to be cases where only one HLV is present?

[gregns1]

Only both HLV's will be here, if one is missing it may be given an implicit HLV or we fall back to rev tree conflict handling.

[torcolvin]

What are the cases where this gets hit vs the end of this function

[gregns1]

The aim here was to keep in sync with current existing conflict resolution functions.

[torcolvin]

LGTM, I think we can sort out the other stuff in a future PR.

[torcolvin]

nit

Suggested change

	var newHLV *HybridLogicalVector
	newHLV, err = db.resolveHLVConflict(ctx, doc, newDoc, conflictResolver.hlvConflictResolver)
	newHLV, err := db.resolveHLVConflict(ctx, doc, newDoc, conflictResolver.hlvConflictResolver)

[torcolvin]

I think this is fine for now, but is it possible that resolving an HLV conflict will also mean that affects a revtree as well?

I suspect it will need to call

sync_gateway/db/crud.go

Lines 1824 to 1862 in 02429a5

	// Note: not setting expiry, as syncData.expiry is reference only and isn't guaranteed to match the bucket doc expiry
	remoteDoc.RemoveBody()
	remoteDoc.Deleted = localDoc.IsDeleted()
	remoteDoc.SetAttachments(localDoc.Attachments().ShallowCopy())
	// If the local doc had attachments, any with revpos more recent than the common ancestor will need
	// to have their revpos updated when we rewrite the rev as a child of the remote branch.
	if remoteDoc.Attachments() != nil {
	// Identify generation of common ancestor and new rev
	commonAncestorRevID := localDoc.SyncData.History.findAncestorFromSet(localDoc.GetRevTreeID(), docHistory)
	commonAncestorGen := 0
	if commonAncestorRevID != "" {
	commonAncestorGen, _ = ParseRevID(ctx, commonAncestorRevID)
	}
	newRevIDGen, _ := ParseRevID(ctx, newRevID)
	// If attachment revpos is older than common ancestor, or common ancestor doesn't exist, set attachment's
	// revpos to the generation of newRevID (i.e. treat as previously unknown to this revtree branch)
	for _, value := range remoteDoc.Attachments() {
	attachmentMeta, ok := value.(map[string]interface{})
	if !ok {
	base.WarnfCtx(ctx, "Unable to parse attachment meta during conflict resolution for %s/%s: %v", base.UD(localDoc.ID), localDoc.SyncData.GetRevTreeID(), value)
	continue
	}
	revpos, _ := base.ToInt64(attachmentMeta["revpos"])
	if revpos > int64(commonAncestorGen) || commonAncestorGen == 0 {
	attachmentMeta["revpos"] = newRevIDGen
	}
	}
	}
	remoteDoc._rawBody = docBodyBytes
	// Tombstone the local revision
	localRevID := localDoc.GetRevTreeID()
	tombstoneRevID, tombstoneErr := db.tombstoneActiveRevision(ctx, localDoc, localRevID)
	if tombstoneErr != nil {
	return "", nil, tombstoneErr
	}

as well.

[gregns1]

This will be taken care of in CBG-4791