1.39.0 - 2025-10-07
Added
- Spark: Normalize dataset names with configurable trimmers
#3996@pawel-big-lebowski Add configurable dataset name normalization with support for date patterns, key-value pairs, and S3 location detection to enable proper dataset subsetting. - Spark: Add missing facets in inputs for Databricks Unity Catalog
#4057@kchledowski Add missing input symlink facets for Databricks Unity Catalog tables.
Changed
- Spark: Refactor tests for dependency collector
#4058@kchledowski Refactor column-level lineage dependency collector tests for better organization and maintainability.
Fixed
- Spec: Fix typo in iceberg commit report facet spec file
#4069@fm100 Fix typo in IcebergCommitReportOutputDatasetFacet property name. - Spark: Fix dataset trimming for CLL inputs
#4061@pawel-big-lebowski Fix dataset name trimming for column-level lineage inputs. - Python: Remove numpy import
#4062@kacpermuda Remove unnecessary numpy import from Python client.
Removed
- Dagster: Remove Dagster integration
#3844@kacpermuda Remove Dagster integration from the repository.