Scale audit workers to achieve target node vetting times

[shaupt131]

Background:

Using the new scalable node auditing system, we are looking to scale to a point where we are vetting new nodes within 14-21 days. Historically (2022) it was taking us on average 60 days to vet a new node on US1, 58 on EU1, and 86 on AP1.

Historical data: https://redash.datasci.storj.io/dashboard/vetting?p_FILTER=created_at&p_PERIOD=month

Acceptance Criteria:

• Double the number of audit workers/pods currently running
  • Currently 4 workers per pod, 1 pod per satellite
• Let it sit as-is for ~3 weeks to gather new node vetting time metrics
• Confirm we are hitting our new target vetting times
  • If not, bring it to the project team to determine how far off we are and what additional changes should be made

[ethanadams]

PR https://github.com/storj/infra/pull/3662

[ethanadams]

Deployed

[storjrobot]

This issue has been mentioned on Storj Community Forum (official). There might be relevant details there:

https://forum.storj.io/t/lets-address-the-held-amount-problem/21927/26