Log Spam from Riak services in Traffic Ops

[ocket8888]

I'm submitting a ...

• bug report

Traffic Control components affected ...

• Traffic Ops

Current behavior:

Traffic Ops will sometimes emit the error:

ERROR: riak_services.go:343: 2020-11-17T14:10:49.716927165Z: getting riak servers: querying riak servers: sql: transaction has already been committed or rolled back

when encountering an error in Riak-related endpoint logic. It's probably calling HandleErr without returning.

Expected behavior:

When a fatal error occurs, handling should end and no further attempts should be made to roll back the linked transaction.

Minimal reproduction of the problem with instructions:

I think you can actually just see it in the default CDN-in-a-Box setup logs.

[rob05c]

It's probably calling HandleErr without returning.

If it is, you should see this in the logs:
https://github.com/apache/trafficcontrol/blob/master/traffic_ops/traffic_ops_golang/api/api.go#L134

called after a write already occurred! Not double-writing!

If you don't see that, I think there's a good chance it isn't actually calling api.HandleErr, api.WriteResp, or any other func that commits the transaction, and that the transaction hasn't actually had COMMIT or ROLLBACK called.

I've seen that transaction has already been committed or rolled back in the past, after strange query errors. Certain bad queries -- which didn't commit or rollback -- seem to cause all future queries on the same transaction to return that. I don't fully understand why.

[ocket8888]

Riak KV as a backend for Traffic Vault is deprecated, so we won't be fixing this.