standalone backingstore agent

[guymguym]

Explain the changes

1. add a standalone backingstore program.
2. specify a port and a path on command line args for each backingstore.
3. choose the agent ip based on the server address for easy cluster networks.
4. allow agent request auth when protocol is not only n2n to support direct tcp connect.

Issues: Fixed #xxx / Gap #xxx

1. NA

Testing Instructions:

1. NA

[shirady]

@guymguym, I looked at the Ceph tests that are failing - in all the attempts the number is 188 failing tests (consistent).
Do you know why it happens?

[liranmauda]

LGTM,
@guymguym with this changes, can we completely drop the ICE, or do we miss something?

[guymguym]

I looked at the Ceph tests that are failing - in all the attempts the number is 188 failing tests (consistent). Do you know why it happens?

@shirady yeah it's a problem in my code - I could see it in the agent log there (in the diagnostics tar). just quickly pasting here for tracking -

Apr-3 6:57:10.132 [Agent/22] [ERROR] Agent.noobaa-default-backing-store-noobaa-pod-3dca8ca8-noobaa_storage-71ac1f18:: agent api not allowed for protocol null connection *(l64zixvu) allowed protocol n2n:
Apr-3 6:57:10.132 [Agent/22] [ERROR] CONSOLE:: RPC._on_request: ERROR srv block_store_api.write_block reqid 9@n2n://642a72762237990029df0496(ln1gnnab) connid *(l64zixvu) Error: agent api not allowed for protocol null
    at Agent._authenticate_agent_api (/root/node_modules/noobaa-core/src/agent/agent.js:640:19)
    at Agent.rpc.register_service.middleware (/root/node_modules/noobaa-core/src/agent/agent.js:190:46)
    at RPC._on_request (/root/node_modules/noobaa-core/src/rpc/rpc.js:323:27)
    at /root/node_modules/noobaa-core/src/rpc/rpc.js:734:38
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 

[guymguym]

with this changes, can we completely drop the ICE, or do we miss something?

@liranmauda not yet - this does allow us to bypass ice and use a fixed AGENT_RPC_PORT and AGENT_RPC_PROTOCOL=tcp, however I still did not fix the callers to send auth_token and validate it in the agents.

[shirady]

@guymguym Did you notice that the build of noobaa tester takes about an hour?

[guymguym]

Did you notice that the build of noobaa tester takes about an hour?

@shirady I don't think it was related to the PR - I see that after the merge new build was regular so perhaps its an unrelated load - https://github.com/noobaa/noobaa-core/actions/runs/4608283705