Document crdb_internal.force_retry() and how to use it to test transaction retry scenarios

[vy-ton]

Even internally, we've recieved feedback to document the expected behavior of crdb_internal.force_retry().

In addition, crdb_internal.force_retry() usage within our third-party tool examples makes it difficult to explain just transaction retry to users. Ideally, we would extract transaction retry developer recipe.

[ericharmeling]

The help text for crdb_internal.force_retry():

demo@127.0.0.1:26257/defaultdb> \hf crdb_internal.force_retry
Function:    crdb_internal.force_retry
Category:    System info
This function is used only by CockroachDB's developers for testing purposes.

Signature
crdb_internal.force_retry(val: interval) -> int

See also:
  https://www.cockroachlabs.com/docs/v21.1/functions-and-operators.html

The generated docs for crdb_internal.force_retry():

According to this description, we shouldn't be encouraging users to use crdb_internal.force_retry() at all.

We could document crdb_internal.force_retry() usage more explicitly (and then update the helptext/generated function docs). But I'm wondering if we want to remove crdb_internal.force_retry() from our examples entirely.

I don't find the function's behavior very intuitive. @timveil seems to agree.

@ajwerner, @rafiss : thoughts? How necessary/helpful is this function for users looking to write retry logic into their applications?

[timveil]

yes yes yes... i'd love to find another way to more simply trigger retry behavior so developers can test their application's behavior. the current usage of this function complicates our examples and is not intuitive. truthfully i still don't understand how to integrate its usage into code examples in a coherent way

[rafiss]

Let's just finalize cockroachdb/cockroach#45224 once and for all. It lets you set a session variable to force txn retriy errors.

Based on the Slack convo here, we should be able to https://cockroachlabs.slack.com/archives/C04U1BTF8/p1609860010064800?thread_ts=1600356504.179000&cid=C04U1BTF8

cc @vy-ton would you agree that we should do the PR I linked instead of documenting this internal function?

[vy-ton]

Yes, the PR Rafi linked is a 21.2 roadmap item.

I created this docs issue as a stopgap until we could provide the session variable. Once the session variable exists, we can modify our examples to stop using crdb_internal.force_retry

I'm not opposed to closing this. Although the better testing won't be available until 21.2

[timveil]

Happy to test as much as I can. I have a bunch of retry code ready to go.

[ericharmeling]

Closing this issue for now.

I assume cockroachdb/cockroach#45224 will be backported, since it was written on 20.2 in the first place. As Vy said, after that's merged, we can coordinate on updating the samples.