HCK-15458: dbutils with fallback

[chulanovskyi-bs]

Content

• Implemented the fallback strategy of fetching tables info for large databases.

Technical details

• Replaced print with dbutils.notebook.exit handler, which doesn't trimm the the content in between.
• However, even with that, the maximum transferable “window” for Databricks command result is usually around 10–20 MB, depending on runtime. It's not strcitly documented, but was obtained experimentally. When we hit that limit - the fallback would be to fetch each table columns partially, by batch.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud