Skip to main content
Version: Next

1.54.0 - 2026-10-12

Added​

  • Client/Java: Support HTTP proxies #4998 @jakub-moravec Routes HTTP events through JVM proxy settings or an explicit transport proxy, with configurable hosts that bypass the proxy.
  • Client/Java: Support Unix domain socket URLs #4920 @pawel-big-lebowski Delivers HTTP events through unix:// collector sockets on Java 16 or later, using the existing endpoint, authentication, and compression settings.
  • Client/Python: Add custom CA certificates and mutual TLS #4983 @srujankgandla Adds sslContext settings for a CA bundle, client certificate, and client key to the HTTP transport.
  • Client/Python: Detect PR numbers in Bitbucket Pipelines and Azure Pipelines #5041 @himakolavennu Populates pullRequestNumber in the source code location facet from Bitbucket and Azure Pipelines environment variables.
  • Clients: Add OAuth 2.0 client credentials authentication #4935 @rahul-madaan Adds the oauth2_client_credentials HTTP authentication provider to the Java and Python clients, with token caching and refresh.
  • dbt: Add IBM watsonx.data Presto adapter support #4978 @aseelert Recognizes watsonx_presto profiles and emits datasets in the same presto://host:port namespace as PrestoDB.
  • dbt: Add PrestoDB adapter support #4969 @aseelert Recognizes presto profiles and derives dataset namespaces from the Presto host and port.
  • Spark: Add column lineage for CREATE and REPLACE TABLE AS SELECT #4984 @ujwalkumar-meesho Traverses childless CTAS and RTAS query plans and nested operators to capture complete column dependencies on Spark 3.5 and Databricks runtimes.
  • Spark: Emit outputs for relation-backed streaming writes #4908 @MSDehghan Captures catalog-aware output datasets for non-Kafka Structured Streaming V2 writes, including Iceberg sinks, on Spark 3.2 or later.
  • Spec: Add capture timestamps to data quality metrics #4936 @pawel-big-lebowski Adds optional captureDate to DataQualityMetricsDatasetFacet to identify when the reported dataset metrics were captured.

Changed​

  • Spark: Add target tables to metadata-only DELETE job names #5033 @MSDehghan Includes the target table in supported metadata-only DELETE job names so deletes against different tables produce distinct jobs.
  • SQL: Upgrade sqlparser and preserve lineage for new SQL forms #5046 @mobuchowski Upgrades sqlparser to 0.63.0, captures dependencies in aggregate WHERE predicates, ESCAPE expressions, IS JSON operands, and Redshift expression unpivoting, and explicitly rejects unsupported SQL pipe queries.

Fixed​

  • Client/Go: Accept all successful HTTP responses #4939 @rahul-madaan Treats every HTTP 2xx response as success, including 202 Accepted responses from consumers that queue events.
  • Client/Java: Ignore historical GC time in circuit breaker checks #4956 @Daksha1611 Initializes the garbage collection baseline correctly so GC time accumulated before the first check does not trigger the circuit breaker.
  • Client/Java: Preserve fully qualified JDBC table names #5044 @MSDehghan Avoids prepending the JDBC URL database to table names that already include a database, including SQL Server cross-database references.
  • Client/Python: Serialize dates and UUIDs as canonical strings #4942 @ptimizeroracle Serializes datetime and date values with ISO formatting and UUIDs as strings instead of non-serializable placeholders in events and custom facets.
  • dbt: Skip exposure job events and handle null Datadog routing fields #4990 @himakolavennu Skips job events for non-executable exposures while preserving their dataset metadata, and handles null integration and job type values in Datadog routing.
  • Flink: Preserve Kafka datasets with unsupported deserializers #4966 @josemasa13 Retains Kafka input dataset identities when a custom deserializer cannot provide a schema facet.
  • Spark: Avoid reusing Iceberg commit reports across runs #5030 @MSDehghan Removes matched commit reports from the shared catalog reporter and caches them per run so later runs do not reuse reports already claimed by an earlier run.
  • Spark: Avoid scanning dynamic file indexes for input statistics #4977 @MSDehghan Uses cached file listings or stored catalog statistics to avoid starting Spark jobs on the listener thread while collecting optional input statistics.
  • Spark: Cancel facet tasks when event processing is interrupted #4922 @mobuchowski Cancels pending dataset or facet work when its waiting thread is interrupted, reducing prolonged memory retention and occupied executor threads.
  • Spark: Capture remaining properties when one configured key is unset #5054 @sg0709 Handles each capturedProperties lookup separately so a missing runtime configuration key does not discard other available properties.
  • Spark: Derive dataset lifecycle changes from the save mode #5031 @MSDehghan Reports CREATE and OVERWRITE only for the corresponding save modes, avoiding incorrect lifecycle facets on JDBC, Delta path, and generic provider appends.
  • Spark: Include computed MERGE assignments in column lineage #4991 @MSDehghan Captures source dependencies for casts, not-null checks, functions, and arithmetic in MERGE assignments instead of retaining only bare column references.
  • Spark: Inject Iceberg metrics reporters when catalog caching is disabled #5013 @MSDehghan Registers metrics reporters for uncached Iceberg catalogs so commit report facets can be emitted with cache-enabled=false.
  • Spark: Isolate facet builder failures #5023 @MSDehghan Continues building events and other facets when a facet builder throws an exception or linkage error.
  • Spark: Keep V2 datasets when version lookup fails #5020 @MSDehghan Emits DataSourceV2 datasets without a version facet when the catalog version lookup fails, preserving the other datasets in the event.
  • Spark: Preserve Delta lineage for top-level adaptive queries #4932 @mobuchowski Uses root execution IDs on Spark 3.4 or later to retain user queries with Adaptive Query Execution while filtering nested Delta executions.
  • Spark: Preserve JDBC inputs in joins following subqueries #5014 @MSDehghan Parses the full dbtable expression when a parenthesized subquery is followed by a join, retaining all input tables.
  • Spark: Prevent late task metrics retention #4921 @mobuchowski Ignores task metrics for stages with no active job owner, preventing orphaned metrics from accumulating after job cancellation.
  • Spark: Retain matching Iceberg metrics reports across catalogs #5009 @MSDehghan Stops searching after finding a matching scan or commit report so a later catalog's empty result cannot discard it.
  • Spark: Skip unsupported Iceberg metrics catalogs safely #4952 @MSDehghan Skips unsupported or missing Iceberg catalogs during metrics reporter injection instead of failing lineage event construction.
  • Spark: Support Iceberg Glue catalogs configured with type=glue #5037 @alokr-dhub Recognizes the Glue catalog shorthand and emits the correct AWS Glue table identity and TABLE symlink.
  • Spark: Tolerate duplicate files in Iceberg input statistics #5022 @MSDehghan Counts each data file once when it appears in multiple scan tasks, preventing duplicate-key errors from dropping input datasets.
  • Spark: Use qualified JDBC names when matching column lineage inputs #4992 @MSDehghan Uses consistent schema-qualified dataset identifiers when matching JDBC query columns to input datasets.
  • SQL: Handle BigQuery UNNEST and UNPIVOT table factors #4962 @MostafaAbdelsalam11 Preserves query lineage around UNNEST expressions and visits the source table and alias of UNPIVOT instead of failing the entire query.
  • SQL: Include QUALIFY subquery inputs #4915 @mattfaltyn Includes tables referenced only by nested QUALIFY subqueries in SQL table lineage.

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2026 contributors to the OpenLineage project