messaging_service: Clear list of clients on shutdown

When messaging_service shuts down it first sets _shutting_down to true
and proceeds with stopping clients and servers. Stopping clients, in
turn, is calling client.stop() on each.

Setting _shutting_down is used in two places.

First, when a client is stopped it may happen that it's in the middle of
some operation, which may result in call to remove_error_rpc_client()
and not to call .stop() for the second time it just does nothing if the
shutdown flag is set (see 357c91a).

Second, get_rpc_client() asserts that this flag is not set, so once
shutdown started it can make sure that it will call .stop() on _all_
clients and no new ones would appear in parallel.

However, after shutdown() is complete the _clients vector of maps
remains intact even though all clients from it are stopped. This is not
very debugging-friendly, the clients are better be removed on shutdown.

fixes: #14624

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

Closes #14632

message/messaging_service.cc

@@ -485,6 +485,11 @@ future<> messaging_service::stop_client() {
             return c.second.rpc_client->stop().then([addr = c.first] {
                 mlogger.info("Stopping client for address: {} - Done", addr);
             });
+        }).finally([&m] {
+            // no new clients should be added by get_rpc_client(), as it
+            // asserts that _shutting_down is true
+            m.clear();
+            mlogger.info("Stopped clients");
         });
     });
 }