Skip to main content
Version: 1.34.0

1.34.0 - 2025-06-18

Added

  • Hive: Integration added. #3555 @tnazarew with @ddebowczyk92, @jphalip
    Added OpenLineage Hive integration
  • Spark: Support dynamic frames. #3691 @pawel-big-lebowski
    Support lineage extraction from UnionRdd and NewHadoopRDD, which makes dynamic frames docker based test passing.
  • Hive: Add hive_query facet #3781 @dolfinus
    Adds hive_query facet for Hive integration
  • Hive: Add job sql facet #3777 @dolfinus
    Adds job sql facet for Hive integration
  • Hive: Add hive session facet #3786 @dolfinus
    Adds hive session facet for Hive integration
  • Java: Add Location Symlink type #3717 @tnazarew
    Add new symlink type representing physical location of dataset
  • Spark: Smart debug facet. #3715 @pawel-big-lebowski
    Automatically turn on debug facet in case of spark connector anomalies detected
  • Spark: Add support for Big Query Metastore catalog type #3760 @ddebowczyk92
    Adds support for BigQuery Metastore catalog in Spark integration
  • dbt: Add DbtRun facet #3738 @dolfinus
    Adds DbtRun facet for tracking dbt run information
  • dbt: Initial support for Clickhouse #3739 @dolfinus
    Adds initial support for ClickHouse in dbt integration
  • dbt: Add processing_engine facet #3725 @dolfinus
    Adds processing_engine facet for dbt integration
  • Flink: Add facet with Flink jobId #3744 @dolfinus
    Adds facet containing Flink job ID information
  • Flink: Add processing_engine facet #3726 @dolfinus
    Adds processing_engine facet for Flink integration
  • JDBC: Column level lineage for jdbc queries load #3763 @pawel-big-lebowski
    Adds column-level lineage support for JDBC queries for Spark with single input table
  • Spec: Add contentType to documentation facet #3748 @dolfinus
    Adds contentType field to documentation facet specification

Changed

  • Airflow: Remove Airflow < 2.5.0 support #3669 @kacpermuda
    Drops support for Airflow versions below 2.5.0
  • dbt: Use adapter rows_affected as outputStatistics #3731 @dolfinus
    Uses adapter's rows_affected for output statistics instead of custom calculation
  • dbt: Move facets from processor module #3713 @dolfinus
    Refactors dbt facets organization by moving them from processor module
  • Java: Speedup generateNewUUID #3754 @dolfinus
    Improves performance of UUID generation in Java client
  • Java: Make UUIDUtils.generateStaticUUID random part more variative #3709 @dolfinus
    Increases randomness in static UUID generation
  • Java: Add log if load from yaml fails #3766 @mvitale
    Adds logging when YAML configuration loading fails
  • Spark: Update Spark 4 dependency to 4.0.0 (remove -preview1 suffix) #3751 @ddebowczyk92
    Updates Spark 4 dependency to final 4.0.0 release
  • Spark: Disable module metadata file generation #3785 @ddebowczyk92
    Disables generation of module metadata files in Spark integration
  • Python: Use attr.define instead of attr.s #3776 @kacpermuda
    Modernizes Python code to use newer attrs API
  • Proxy: Remove native proxy #3680 @mobuchowski
    Removes the native proxy implementation

Fixed

  • Spark: Fix missing table path in InsertIntoHadoopFsRelationCommand #3773 @dolfinus
    Fixes issue where table path was missing in InsertIntoHadoopFsRelationCommand
  • BigQuery: Filter temp inner jobs for bigquery indirect mode #3722 @pawel-big-lebowski
    Filters out temporary inner jobs in BigQuery indirect mode
  • dbt: dbt-ol should not error on job complete if there is no start event #3749 @mobuchowski
    Prevents errors when job completion occurs without corresponding start event
  • Flink: Do not hide OpenLineage config parsing errors #3724 @dolfinus
    Improves error visibility for OpenLineage configuration parsing in Flink
  • Java: Prevent original events from being mutated in TransformTransport #3728 @JDarDagran
    Ensures original events remain immutable during transport transformations
  • Java: Fix visibility of GcpLineageTransportConfig.Mode #3762 @ngorchakova
    Corrects visibility modifier for GCP transport configuration mode