db: fix SINGLEDEL, MERGE + DEL interaction

[jbowens]

Previously, when a MERGE consumed a deletion tombstone during a compaction, the MERGE key was transformed into a SET. This SET lost knowledge of the previous deletion. Previously, a subsequent SingleDelete (intended to remove the MERGE key) could elide the SET, resurrecting internal keys that were deleted by the consumed deletion tombstone.

This commit adjusts the compaction iterator to produce a SETWITHDEL (see #1255) that preserves knowledge of the consumed deletion tombstone. This ensures that a SINGLEDEL that meets the merged key yields a DEL.

[cockroach-teamcity]

This change is 

[sumeerbhola]

what motivated trying to support SINGLEDEL for MERGE?

Reviewable status: 0 of 4 files reviewed, all discussions resolved

[jbowens]

It wasn't clear to me that we weren't supporting it. The Writer.SingleDelete comments use loose language that doesn't explicitly say Set.

The SingleDelete operation is akin to "anti-matter" and will only delete the most recently written version for a key.

pebble/db.go

Lines 124 to 142 in 51fca96

	// SingleDelete is similar to Delete in that it deletes the value for the given key. Like Delete,
	// it is a blind operation that will succeed even if the given key does not exist.
	//
	// WARNING: Undefined (non-deterministic) behavior will result if a key is overwritten and
	// then deleted using SingleDelete. The record may appear deleted immediately, but be
	// resurrected at a later time after compactions have been performed. Or the record may
	// be deleted permanently. A Delete operation lays down a "tombstone" which shadows all
	// previous versions of a key. The SingleDelete operation is akin to "anti-matter" and will
	// only delete the most recently written version for a key. These different semantics allow
	// the DB to avoid propagating a SingleDelete operation during a compaction as soon as the
	// corresponding Set operation is encountered. These semantics require extreme care to handle
	// properly. Only use if you have a workload where the performance gain is critical and you
	// can guarantee that a record is written once and then deleted once.
	//
	// SingleDelete is internally transformed into a Delete if the most recent record for a key is either
	// a Merge or Delete record.
	//
	// It is safe to modify the contents of the arguments after SingleDelete returns.
	SingleDelete(key []byte, o *WriteOptions) error

Reviewable status: 0 of 4 files reviewed, all discussions resolved

[jbowens]

This part of the comment makes it seem like it is supported.

SingleDelete is internally transformed into a Delete if the most recent record for a key is either a Merge or Delete record.

The semantics that I thought we supported was it's okay to SingleDelete a key that has been written using Merge, as long as it's only been written once since the last Delete. There's really no point to using Merge in that scenario, because you presumably already know that your single Merge key will not be merged with any existing keys. I'm okay with either semantics.

Reviewable status: 0 of 4 files reviewed, all discussions resolved

[sumeerbhola]

Reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jbowens)

---

testdata/compaction_iter_delete_sized line 1883 at r1 (raw file):

.

# When a MERGE and a DEL[SIZED] meet in a comapction, a SETWITHDEL key (NOT a

nit: compaction

[jbowens]

TFTR!