Handle CRDB Retry responses

[ecordell]

See: cockroachlabs.com/docs/v21.1/transactions#client-side-intervention

Here's an example of spicedb logs when a retry isn't handled:

5:04PM ERR Exec args=[] err="ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=caf1be11 key=/Table/54/2/\"QRdiUVdMQmkyjQjX\"/\"user\"/\"...\"/\"resource\"/\"direct\"/\"thegoods\"/0 pri=0.04592771 epo=0 ts=1631394282.849166225,2 min=1631394148.811930011,0 seq=6} lock=true stat=PENDING rts=1631394148.811930011,0 wto=false gul=1631394149.311930011,0 (SQLSTATE 40001)" module=pgx sql=commit
5:04PM ERR finished server unary call error="unable to write tuples: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=caf1be11 key=/Table/54/2/\"QRdiUVdMQmkyjQjX\"/\"user\"/\"...\"/\"resource\"/\"direct\"/\"thegoods\"/0 pri=0.04592771 epo=0 ts=1631394282.849166225,2 min=1631394148.811930011,0 seq=6} lock=true stat=PENDING rts=1631394148.811930011,0 wto=false gul=1631394149.311930011,0 (SQLSTATE 40001)" grpc.code=Unknown grpc.method=Write grpc.method_type=unary grpc.service=authzed.api.v0.ACLService grpc.start_time=2021-09-11T17:02:29-04:00 grpc.time_ms=3712.738 kind=server system=grpc