lakehouse federation

• definition in Azure
• enable users and systems to run queries against multiple data sources without needing to migrate all data to a unified system.

query federation

Unity Catalog queries are pushed down to the foreign database using JDBC.

• The query is run both in Databricks and using remote compute.
• Use case: ad hoc reporting or proof-of-concept access to operational data stored in external databases.

Supported

• MySQL
• PostgreSQL
• Teradata
• Oracle
• Amazon Redshift
• Salesforce Data Cloud
• Snowflake
• Microsoft SQL Server
• Azure Synapse (SQL Data Warehouse)
• Google BigQuery
• Databricks

catalog federation

• Queries directly access the foreign table in object storage.
• Catalog federation is available for platforms that support direct access to their catalog and support open table formats
• The query is only run on Databricks compute
  • more cost-effective and performance-optimized than query federation.
• usecases:
  • migrate to Unity Catalog: need to incrementally phase in data managed from a foreign catalog.
  • a long-term co-exist hybrid model

Supported

• External Hive metastore
• AWS Glue metastore
  • Please note that corresponding page in Azure does not exist
  • Fall back to External Hive metastore case.
• Salesforce Data Cloud
• Snowflake