Memory Usage of StatisticsAwareJdbcClient

[Wetbikeboy2500]

I have a few SQL server instances that have a few hundred databases each that I am setting up as catalogs for trino. I noticed that during the startup of the coordinator, it was significantly increasing in memory and would log warnings for large GC cleanups.

I investigated using the trino-server-dev setup and created 1000 catalogs with:

mkdir -p catalog
for i in {1..1000}; do
cat <<EOF > catalog/sqlserver_$i.properties
connector.name=sqlserver
connection-url=jdbc:sqlserver://localhost:1433;databaseName=db_$i
connection-user=sa
connection-password=YourPassword123
EOF
done

I ran the profiler and saw:

The ramp up in memory was when all the catalogs where being loaded in. A memory dump shows the JMXMBeamServer taking up 2.4GB

I decided to hack around the StatisticsAwareJdbcClient and remove it along with the MBeam setup from the JdbcDiagnosticModule and memory barely climbed.

I also was curious if removing the StatisticsAwareJdbcClient helped in performance for sequential queries as it would not be tracking so much data and saw requests go from 25.16 req/sec to 34.43 req/sec using a SqlServerQueryRunner set up with one catalog.

Is it possible to disable the StatisticsAwareJdbcClient per catalog? I think best case would be that I could turn on StatisticsAwareJdbcClient for a sub-set of my catalogs through a catalog config option to monitor them, but overall monitoring can be done more efficiently at the applications on top of trino or at the databases themselves.