You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
executed in a shared Iceberg worker pool. It's shared among queries so high concurrency of metadata-heavy queries can make things worse. And this thread pool is not managed by Trino so its memory usage is not tracked.
so, we should allow this property to be passed from the catalog, so that it will not be set statically.
The text was updated successfully, but these errors were encountered:
I think this needs to be fixed in the Iceberg library to allow per-catalog configuration rather than relying on a global system property. I don't see a way to fix this on the Trino side.
There is a planWith(ExecutorService executorService) API available, which we are using in IcebergSplitSource (configurable though Make Iceberg split manager threads configurable).
We could extend it's usage to all callers of planFiles to control number of threads on Trino side.
related issue: #11708 comment: #11708 (comment)
TableScan::planFiles
executed in a shared Iceberg worker pool. It's shared among queries so high concurrency of metadata-heavy queries can make things worse. And this thread pool is not managed by Trino so its memory usage is not tracked.
so, we should allow this property to be passed from the catalog, so that it will not be set statically.
The text was updated successfully, but these errors were encountered: