oasis-node crashes, if metrics in push mode and PGW crashes

[matevz]

Currently, if oasis-node is run in metrics push mode and push gateway unexpectedly crashes, so does oasis-node.

Improve oasis-node Prometheus pusher code as follows:

1. If PGW is not responding, try to reconnect.
2. When starting oasis-node in push mode and if PGW is not accessible, try to reconnect instead of failing to start.

[kostko]

How does it crash, any stack traces?

[Yawning]

If PGW is not responding, try to reconnect.

Make the upstream code not brain-damaged then.

https://pkg.go.dev/github.com/prometheus/client_golang/prometheus/push?tab=doc#Pusher.Push

More specifically, our code assumes that each call to Pusher.Push can succeed or fail independently, which is incorrect. Once a call to Push fails, it will continue to fail, returning the same error.