Skip to main content
Version: Next

1.35.0 - 2025-07-11

Added

  • Spark: Include spark_applicationDetails facet to all events #3848 @dolfinus
    Add spark_applicationDetails facet to all OpenLineage events emitted by the Spark integration
  • Spark: Support additional data in Catalog facet #3850 @ddebowczyk92
    Adds support additional data in Catalog in Spark integration
  • Spark: disable connector by Spark config parameter #3880 @pawel-big-lebowski
    Add spark.openlineage.disabled entry to disable OpenLineage integration through Spark config parameters
  • Spark: Fine-grained timeout config #3779 @pawel-big-lebowski
    Add extra timeout options to emit incomplete OpenLineage events in case of timeout when building facets. See buildDatasetsTimePercentage and facetsBuildingTimePercentage in docs for more details
  • Python: Asynchronous HTTP transport implementation #3812 @mobuchowski
    Adds high-performance asynchronous HTTP transport with event ordering guarantees, configurable concurrency, and comprehensive error handling. Features START-before-completion event ordering, bounded queues, and real-time statistics
  • dbt: Add DbtRun facet to dbt run events #3764 @dolfinus
    Adds DbtRun facet for tracking dbt run information
  • Python: Add continue_on_success and sorting transport in CompositeTransport #3829 @kacpermuda
    Adds configuration options for CompositeTransport to control behavior and ordering
  • Hive: Add jobType facet #3789 @dolfinus
    Adds jobType facet to Hive integration
  • Hive: Add dialect=hive to SqlJobFacet #3863 @dolfinus
    Adds dialect field to SqlJobFacet for Hive integration
  • Java: Add Debezium job naming convention #3834 @mvitale
    Adds standardized job naming convention for Debezium integration
  • Spec: SqlJobFacet now contains dialect #3819 @mobuchowski
    Adds dialect field to SqlJobFacet specification
  • Spec: Formalize job naming #3826 @ddebowczyk92
    Formalizes job naming conventions in the specification
  • Spec: Formalize dataset naming #3775 @ddebowczyk92
    Formalizes dataset naming conventions in the specification
  • Website: Add Collibra as consumer in ecosystem page #3732 @marccampa
    Adds Collibra to the ecosystem page as an OpenLineage consumer

Changed

  • Spark: Use Hive as default Iceberg catalog implementation #3858 @dolfinus
    Updates Spark integration to use Hive as the default catalog implementation for Iceberg tables
  • Spark: Replace weak hash-map with a map with weak keys and entries #3856 @ddebowczyk92
    Improves memory management in Spark integration by replacing weak hash-map implementation
  • Spark: Support latest databricks runtime #3811 @pawel-big-lebowski
    Updates Spark integration to support the latest Databricks runtime
  • Python: Remove transport.wait_for_completion() #3881 @dolfinus
    Removes wait_for_completion() method from Python transport interface
  • Python: Reuse session in sync HttpTransport #3843 @dolfinus
    Improves performance by reusing HTTP sessions in synchronous transport
  • Python: Implement Transport.close() for Datazone and Kinesis #3857 @dolfinus
    Adds proper cleanup methods for Datazone and Kinesis transports
  • Python: Implement TransformTransport.close #3855 @dolfinus
    Adds cleanup method for TransformTransport
  • Python: Implement KafkaTransport.wait_for_completion() and .close() #3838 @dolfinus
    Adds proper cleanup and completion methods for Kafka transport
  • Java: Make CompositeTransport.close() more reliable #3841 @dolfinus
    Improves reliability of CompositeTransport cleanup process
  • Java: Cover OpenLineageClient.close() with tests #3839 @dolfinus
    Adds test coverage for OpenLineageClient cleanup methods
  • Java: Name threads used in Java client #3817 @mobuchowski
    Adds meaningful names to threads used in Java client for better debugging
  • Flink: Close OpenLineageClient in onJobExecuted #3854 @dolfinus
    Ensures proper cleanup of OpenLineageClient in Flink 1.x integration
  • Flink: Fixed a bug incorrectly loading configuration in Event Emitter #3799 @fetta
    Fixes configuration loading issue in Flink Event Emitter
  • dbt: Make invocation_id field optional #3796 @dolfinus
    Makes invocation_id field optional in dbt integration
  • dbt: More resiliency for missing dbt nodes #3836 @mobuchowski
    Improves error handling for missing dbt nodes
  • Hive: Add docker-compose example for local testing #3800 @dolfinus
    Adds docker-compose setup for local Hive integration testing
  • Airflow: Send pending events after Airflow DAG is finished #3849 @dolfinus
    Ensures all pending events are sent after DAG completion in Airflow integration

Fixed

  • dbt: Fix log path, more precise file reading #3793 @mobuchowski
    Improves log file handling in dbt integration
  • dbt: Fix deprecated configs #3859 @kacpermuda
    Replaces deprecated dbt configurations with current alternatives
  • Spark: Fix missing .db suffix in database/namespace location name for BigQueryMetastoreCatalog #3874 @ddebowczyk92
    Fixes database naming issue in BigQuery Metastore catalog implementation
  • Spark: Fix missing output's dataset catalog facet when running CTAS queries on Iceberg tables #3835 @ddebowczyk92
    Fixes missing catalog facet in output datasets for CTAS queries on Iceberg tables
  • Spark: Delta merge with column #3871 @pawel-big-lebowski
    Fixes Delta merge operation handling with column-level lineage
  • Spark: Support delta 3.3.2 #3861 @pawel-big-lebowski
    Adds support for Delta Lake version 3.3.2
  • Spark: Call version utils method when dataset can be identified #3832 @pawel-big-lebowski
    Fixes version handling when dataset can be properly identified
  • Java: Raise error if no events were emitted by composite transport #3853 @kacpermuda
    Adds error handling when CompositeTransport fails to emit any events
  • Java: Fix IntelliJ reload #3825 @mobuchowski
    Fixes IntelliJ project reload issues
  • Java: Fix spotless in hive integration #3806 @mobuchowski
    Fixes code formatting issues in Hive integration
  • Java: Fix missing field in SQL facet test #3830 @mobuchowski
    Fixes missing field in SQL facet test case
  • Rust: Fix new rust formatting warnings #3814 @mobuchowski
    Fixes formatting warnings in Rust code