Performance & Volumes (Cloud Fetch restrictions)

[benoitcayladbx]

Today, the solution uses CloudFetch when possible to handle large result volumes.

In some network configurations, Databricks Apps cannot reach the internal *.storage.cloud.databricks.com hosts used by CloudFetch. To avoid Digital Twin build failures or long delays, we temporarily disable CloudFetch in App mode and fall back to inline Thrift results over the workspace endpoint (already authenticated).

This option can be reverted easily by removing the lines:

    # *** START FIX - CLOUD FETCH WITH DEPLOYED APPS ***
    if self.is_app_mode:
        params["use_cloud_fetch"] = False
    # *** END FIX - CLOUD FETCH WITH DEPLOYED APPS ***

from the get_sql_connection_params() in src/back/core/databricks/DatabricksAuth.py file

To re-enable CloudFetch in Databricks Apps, the app network policy must allow all hostnames used on leg B (not only *.cloud.databricks.com). If that is not possible, the next release will provide an alternative compatible with any network configuration.