Skip to main content
Version: Next

1.48.0 - 2026-06-02

Added​

  • Spark: Add iceberg s3 tables catalog support #4558 @mobuchowski Fix incorrect Glue table symlink attachment when using S3 Tables as the Iceberg catalog by adding dedicated S3 Tables catalog support and preventing Glue fallback when the S3 Tables catalog is active.
  • Spark: Add Lakehouse Hive Catalog handling #4574 @tnazarew Add support for the Lakehouse (formerly BigLake) Hive Catalog used by Spark jobs running on Managed Spark (formerly Dataproc).
  • Spark: Add Snowflake Iceberg REST catalog (Horizon) support #4546 @adnanhemani Add support for the Snowflake Horizon Iceberg REST catalog, enabling the Spark integration to emit correct dataset identifiers with Snowflake namespaces for Iceberg tables managed through Snowflake's REST catalog.

Changed​

  • Client/Java: Upgrade Kinesis Producer Library to 1.0.7 and AWS SDK to 2.44.3 #4536 @W-Ely Upgrade the Amazon Kinesis Producer Library (KPL) from 0.15.12 to 1.0.7 and update AWS SDK dependencies to 2.44.3 to maintain compatibility and pick up upstream fixes.
  • Python: Log if JSON parsing fails for env var #4552 @kacpermuda Add a warning log message when JSON parsing of OpenLineage configuration from environment variables fails, making configuration errors easier to diagnose.
  • Spark: Make iceberg catalog identifiers optional #4557 @mobuchowski Change BaseCatalogTypeHandler.getIdentifier() to return Optional<DatasetIdentifier> instead of a nullable value, making the contract for Iceberg catalog type handlers more explicit and null-safe.
  • Spark: Refactor of Dataset builder #4587 @tnazarew Refactor Spark integration's dataset-building logic from overloaded methods to a more flexible builder pattern, improving extensibility and maintainability.

Fixed​

  • Python: Detect GitHub pull request head refs #4561 @hcthakur2004 Fix pull request number detection in the Python client to recognize refs/pull/<number>/head format (used in GitHub Actions pull_request events) in addition to the existing refs/pull/<number>/merge form.
  • Spark: Fix SnowflakeCatalogTypeHandler.getIdentifier() return type #4571 @mobuchowski Fix SnowflakeCatalogTypeHandler.getIdentifier() to return Optional<DatasetIdentifier> as required by BaseCatalogTypeHandler, correcting a type mismatch introduced when the Snowflake Iceberg REST catalog handler was first added.
  • Spark: Map sqlserver to MsSqlDialect #4556 @dbathie-wtg Map Spark JDBC sqlserver URLs to MsSqlDialect so SQL Server queries using bracketed identifiers ([schema].[table]) are parsed correctly during lineage extraction.