Session pool metrics registered multiple times

[olavloite]

The new metrics that have been added for the SessionPool are added multiple times if multiple different session pools are created by a Spanner instance. This could happen if the user creates database clients for different databases, or if multiple database clients using different SessionPoolOptions are requested. This again will lead to an InvalidArgumentException being thrown. See for example https://source.cloud.google.com/results/invocations/a8dc0694-336f-483b-bb8f-2ee502506a2b/targets/cloud-devrel%2Fjava%2Fjava-docs-samples%2Fjava8%2Fpresubmit/log

[mayurkale22]

Copying discussion from OpenCensus PR census-instrumentation/opencensus-java#2017 (comment):

The only concern that I still have is what it looks like in the dashboards when there are multiple different instances of a SessionPool with the same label values. I understand that it won't cause any errors in the client library, but how is it presented to the user? This situation would occur when an application is opening two connections to the same database with different SessionPoolOptions. This would also mean different values for constant metrics like MaxSessions.

[mayurkale22]

(Thinking out loud) This is a great point. Yes, you're correct, it won't cause any errors in the client library, but as we are having same label values for multiple database clients for the same database (w/ different SessionPoolOptions), the collected metrics would mix with each other and would look messy. Also, it would be challenging/impossible for the users to make any sense of it. I am not sure how to handle this scenario, please let me know if you have any suggestion.

+@rghetia

[rghetia]

One option is to create two more labels.

1. SessionPoolID - UUID for each sessionPool.
2. SessionPoolOptionHash - Hash value computed from all session options.

/1 allows to keep all stats separate.
/2 allows to group stats on the backend (stackdriver) with same options (if hash has no collision).

Users could customize their views to include /1 and/or /2 as necessary.

[olavloite]

One option is to create two more labels.

1. SessionPoolID - UUID for each sessionPool.
2. SessionPoolOptionHash - Hash value computed from all session options.

/1 allows to keep all stats separate.
/2 allows to group stats on the backend (stackdriver) with same options (if hash has no collision).

Users could customize their views to include /1 and/or /2 as necessary.

That sounds reasonable to me. The only thing that concerns me a little bit is that a user will have no idea which SessionPoolID and which SessionPoolOptionHash values belong to which client, so if there's a problem, it will still be very difficult to figure out which client is causing the problem. Or is there some way that a user can set something like an ApplicationID or ClientID in OpenCensus, which indicates 'these metrics are all coming from this application/client'?

[rghetia]

That sounds reasonable to me. The only thing that concerns me a little bit is that a user will have no idea which SessionPoolID and which SessionPoolOptionHash values belong to which client, so if there's a problem, it will still be very difficult to figure out which client is causing the problem. Or is there some way that a user can set something like an ApplicationID or ClientID in OpenCensus, which indicates 'these metrics are all coming from this application/client'?

Ideally, it would be nice if user provides such metadata as part of creating client-connection/session-pool which can then be used as regular label just like SessionPoolID (above). Not sure if such metadata exists.
In the absence of such metadata, application-owners could use constant labels. Not ideal as it requires one more thing to do for the app-owners.