Repository navigation
v1.4.0.6
DataHub Ingestion v1.4.0.6
π¨ Breaking Changes
- Oracle connector URN update β When connecting via
service_nameto a multitenant Oracle database, dataset URNs now use the Pluggable Database (PDB) name instead of the Container Database (CDB) name. Seturn_db_namein your recipe to preserve old URNs. (datahub-project#16396) β @acrylJonny - Python packaging migration β Dependency declarations now use
pyproject.toml(PEP 621).setup.pyremains the source of truth for now but will be deprecated in a future release. (datahub-project#16339) β @kyungsoo-datahub
π New Features
- New Snowplow connector β Ingest metadata from Snowplow analytics pipelines (datahub-project#15735) β @treff7es
- dbt semantic models β Full support for ingesting dbt semantic model metadata (datahub-project#16236) β @alfiyas-datahub
- dbt
convert_urns_to_lowercaseβ Opt-in flag to prevent duplicate entities from mixed-case identifiers on case-insensitive platforms like Snowflake (datahub-project#16358) β @alfiyas-datahub - Snowflake pattern pushdown β Metadata pattern pushdown and table type filtering for improved performance (datahub-project#16100) β @rajatoss
- Trino column-level lineage β Column-level lineage support on
upstreamLineage(datahub-project#16292) β @alfiyas-datahub - Iceberg domain assignment β Ingestion-time domain assignment for Iceberg sources (datahub-project#16443) β @sergey-pozdnyakov-epam
- MongoDB AWS IAM auth β Added
pymongo[aws]extra for AWS IAM authentication (datahub-project#16412) β @javabrett - Kafka Avro validation toggle β Option to disable Avro schema name validation (datahub-project#16310) β @Devarsh23
- Kafka Connect bundled JVM β Bundle JVM via
jdk4pyto remove system Java dependency (datahub-project#16445) β @StanDmitrievAiven - CLI agent improvements β Agent-friendly
datahub graphqlanddatahub initenhancements (datahub-project#16476) β @shirshanka
π Bug Fixes
- Redshift β Boundary-aware segment stitching for query reconstruction (datahub-project#16253) β @kyungsoo-datahub
- Tableau β Apply project filters to embedded datasources when
emit_all_embedded_datasourcesis enabled (datahub-project#16340) β @aviraj-gour - Dagster β Preserve DataJob lineage on failed/canceled runs (datahub-project#16386) β @treff7es
- Snowflake β Quoting fix (datahub-project#16393), map COPY query type to INSERT (datahub-project#16461) β @treff7es
- Teradata β Set DATABASE context for view HELP commands (datahub-project#16208) β @JohnRTurner
- Kafka Connect β Use canonical
mssqlplatform for Debezium SQL Server (datahub-project#16413) β @treff7es - Oracle β Fix profiling crashes and silent table exclusions (datahub-project#16396) β @acrylJonny
- Snowplow β Add missing
cachetoolsdependency (datahub-project#16442) β @treff7es
β‘ Performance
- Snowflake tags β Emulate tag inheritance in-memory to eliminate N+1 queries (datahub-project#16400) β @treff7es
- Dataplex β Streamline ingestion by removing unnecessary entity lookups (datahub-project#16063) β @NehaGslab
π§ Maintenance
- Upgrade to
urllib3v2 (datahub-project#16464) β @sgomezvillamor - Iceberg source recipe examples updated (datahub-project#16417) β @skrydal
π Documentation
- Microsoft Copilot Context Kit guide and misc docs improvements (datahub-project#16452) β @jjoyce0510
- Connector development guide for datahub-skills (datahub-project#16435) β @maggiehays
Contributors: @acrylJonny, @alfiyas-datahub, @aviraj-gour, @Devarsh23, @javabrett, @jjoyce0510, @JohnRTurner, @kyungsoo-datahub, @maggiehays, @NehaGslab, @rajatoss, @sergey-pozdnyakov-epam, @sgomezvillamor, @shirshanka, @skrydal, @StanDmitrievAiven, @treff7es