Skip to main content
Version: Next

1.39.0 - 2025-10-07

Added​

  • Spark: Normalize dataset names with configurable trimmers #3996 @pawel-big-lebowski Add configurable dataset name normalization with support for date patterns, key-value pairs, and S3 location detection to enable proper dataset subsetting.
  • Spark: Add missing facets in inputs for Databricks Unity Catalog #4057 @kchledowski Add missing input symlink facets for Databricks Unity Catalog tables.

Changed​

  • Spark: Refactor tests for dependency collector #4058 @kchledowski Refactor column-level lineage dependency collector tests for better organization and maintainability.

Fixed​

  • Spec: Fix typo in iceberg commit report facet spec file #4069 @fm100 Fix typo in IcebergCommitReportOutputDatasetFacet property name.
  • Spark: Fix dataset trimming for CLL inputs #4061 @pawel-big-lebowski Fix dataset name trimming for column-level lineage inputs.
  • Python: Remove numpy import #4062 @kacpermuda Remove unnecessary numpy import from Python client.

Removed​

  • Dagster: Remove Dagster integration #3844 @kacpermuda Remove Dagster integration from the repository.