Unknown issue with gRPC server running on Google Cloud Run

[kibaekkim]

appfl is running on Google Cloud Run, most of the time.

I observe that the appfl service on the cloud (sometimes) restarts while training, particularly right after the communication of model parameters. I could not find a reason... If this happens, the learning process (i) terminates with error because weights are reset to empty; or (ii) some clients fall into an infinite loop of receiving weights.

[kibaekkim]

This looks relevant: https://stackoverflow.com/questions/65354537/cloud-run-openblas-warnings-and-application-restarts-not-a-cold-start-issue

About OpenBLAS warning: https://cloud.google.com/run/docs/troubleshooting

[kibaekkim]

As our gRPC server is running well with Google Endpoints, this may not be our issue.