Only record sync failures

[vicky-g]

Description

This is so that aborted syncs are not recorded.

Current environment:

127.0.0.1:6379> keys PRIMARY*
1) "PRIMARY_TO_SECONDARY_SYNC_FAILURE::2022-11-30::https://creatornode9.staging.audius.co/::0xa722151fdff6dc5c94803b4712d63730ea6feb08"
2) "PRIMARY_TO_SECONDARY_SYNC_FAILURE::2022-11-30::https://creatornode8.staging.audius.co/::0xa722151fdff6dc5c94803b4712d63730ea6feb08"
127.0.0.1:6379> hgetall PRIMARY_TO_SECONDARY_SYNC_FAILURE::2022-11-30::https://creatornode9.staging.audius.co/::0xa722151fdff6dc5c94803b4712d63730ea6feb08
1) "abort_sync_in_progress"
2) "1"
127.0.0.1:6379> hgetall PRIMARY_TO_SECONDARY_SYNC_FAILURE::2022-11-30::https://creatornode8.staging.audius.co/::0xa722151fdff6dc5c94803b4712d63730ea6feb08
1) "abort_sync_in_progress"
2) "1"

Tests

• forced an abort and observed logs

server logs:

{"name":"audius_creator_node","clusterWorker":"master","hostname":"48a6f6fb4782","pid":136233,"wallet":"0xc3d1d7559356aac64ba2921ec266ce294dcdc981","sync":"secondarySyncFromPrimary","primary":"http://cn3_creator-node_1:4002","syncAllStepsDuration":0,"level":40,"msg":"Sync complete for wallet: 0xc3d1d7559356aac64ba2921ec266ce294dcdc981. Status: Abort. Duration sync: 1. From endpoint http://cn3_creator-node_1:4002. Prometheus result: abort_sync_in_progress","time":"2022-11-30T20:11:27.720Z","v":0,"trace_id":"9bbd50d6597bfbbe801a76854ae98d27","span_id":"4d9f0d0e8f0fbd10","trace_flags":"01","resource.span.name":"_secondarySyncFromPrimary","resource.span.links":[],"resource.span.attributed":{"code.filepath":"/usr/src/app/src/services/sync/secondarySyncFromPrimary.ts","code.function":"_secondarySyncFromPrimary","spid":0,"endpoint":"http://cn2_creator-node_1:4001","result":"abort_sync_in_progress","mode":"default","abort":"Cannot change state of wallet 0xc3d1d7559356aac64ba2921ec266ce294dcdc981. Node sync currently in progress"},"resource.service.name":"content-node","resource.service.spid":0,"resource.service.endpoint":"http://cn2_creator-node_1:4001","logLevel":"warn"}

redis:

127.0.0.1:6379> keys PRIMARY*
(empty list or set)

Monitoring - How will this change be monitored? Are there sufficient logs / alerts?

Monitor SecondarySyncHealthTracker logs

[theoilie]

good catch