1.53.0 - 2026-09-01

Added

• Client/Java: Add configurable retries to the GCP Lineage transport #4759 @tnazarew Exposes GCP Lineage API retry settings through transport configuration and applies them to the underlying client.
• Client/Java: Add configurable retries to the GCS transport #4798 @tnazarew Exposes the lineage producer's retry policy through GcsTransport configuration instead of always using defaults.
• Client/Java: Generate singleton-enum discriminated unions #4778 @mobuchowski Generates typed Jackson union interfaces and concrete variants, enabling correct construction and deserialization of explicit lineage entries.
• Client/Java: Support Oracle TNS connect descriptors #4812 @karthikchundi-commits Parses Oracle RAC and failover TNS descriptors into stable oracle://host:port dataset namespaces.
• Client/Python and Spark: Add Python 3.14 and Spark 4.2 support #4747 @mobuchowski Adds Python 3.14 support and validates the Spark integration on Spark 4.2 with Java 17 and Java 21.
• Client/Python: Add dataset normalization #4465 @kchledowski Adds opt-in normalization of input and output datasets before the Python client emits an event.
• dbt/Athena: Add Glue dataset symlinks #4875 @mobuchowski Adds canonical AWS Glue table identifiers as symlinks for datasets emitted by the dbt Athena integration.
• Flink: Add Kinesis lineage visitors #4861 @fmorillo7694 Aligns SQL and DataStream Kinesis dataset identities and converts Kinesis type metadata into schema facets.
• Spark: Add ClickHouse V2 catalog handler #4878 @MSDehghan Adds dataset identifiers plus catalog and storage facets for reads and writes through the official ClickHouse V2 catalog.
• Spark: Add descriptions to column lineage transformations #4782 @mobuchowski with @tnazarew Captures expression descriptions while resolving Spark column-level lineage transformation chains.
• Spark: Add Lakehouse catalog handler #4797 @tnazarew Adds dedicated handling for Google Lakehouse Catalog identifiers instead of relying on the generic REST catalog path.
• Spark: Emit LOAD DATA source paths as inputs #4848 @mishrasangeeta87 Emits the filesystem path read by LOAD DATA INPATH as an input dataset alongside the target table output.
• Spec: Add explicit lineage facets #4804 @mobuchowski Adds Job and Dataset facets for declaring exact dataset-, field-, and job-level relationships without Cartesian-product inference.
• SQL: Support parenthesized joins #4859 @mattfaltyn Extracts table lineage from nested join expressions while preserving aliases, constraints, and subquery traversal.

Changed

• Client/Python: Replace httpx with httpx2 #4902 @dolfinus Moves AsyncHttpTransport to the maintained httpx2 drop-in replacement and updates related configuration and tests.
• Clients: Enforce dependentRequired in generated models #4819 @mobuchowski Adds Java, Python, and Go validation for schema fields that must be supplied together, including explicit lineage job identities.

Fixed

• Client/Java: Fix JDBC IPv6 custom-port extraction #4612 @matveeysv Parses custom ports after bracketed IPv6 hosts without appending an incorrect default port.
• Client/Java: Make ReducedDataset facet comparison null-safe #4838 @MSDehghan Prevents partition-aware dataset reduction from failing when generated or mocked datasets have null facets.
• Client/Python: Keep the async worker alive after wait #4824 @mattfaltyn Keeps AsyncHttpTransport usable after wait_for_completion() so later events are still delivered.
• Client/Python: Prevent async release-queue deadlock #4882 @mattfaltyn Moves released completion events to a worker-owned backlog so bursts cannot block the sole worker on its bounded queue.
• Client/Python: Queue async completion events atomically #4900 @mattfaltyn Prevents terminal run events from being stranded when their START event completes concurrently.
• Client/Python: Read configuration files as UTF-8 #4509 @hcthakur2004 Avoids locale-dependent decoding failures by reading Python client YAML configuration explicitly as UTF-8.
• dbt: Read OPENLINEAGE_CONTEXT on the local-artifacts path #4808 @chuenchen309 Propagates standardized parent and root run metadata when the dbt wrapper consumes local artifacts.
• dbt: Set the data-quality assertion name on structured-log events #4809 @chuenchen309 Adds the dbt test name to structured-log dataQualityAssertions, matching the run-results processor.
• dbt: Use test_metadata presence when parsing assertions #4777 @kacpermuda Restores short assertion types and column associations for generic tests in dbt manifest v12.
• Great Expectations: Emit a valid producer URI and timezone-aware eventTime #4846 @chuenchen309 Replaces the unresolved producer placeholder and emits RFC 3339 timestamps with a UTC offset.
• Great Expectations: Restore data-quality metrics on 1.x #4733 @chuenchen309 Uses the renamed expectation type field and maps file-size results to bytes, restoring metrics and assertions.
• Great Expectations: Stop forwarding data_context on 1.x #4811 @chuenchen309 Allows OpenLineageValidationAction to construct on Great Expectations 1.x while retaining compatibility with versions that require the argument.
• Java integrations: Upgrade Jackson to 2.18.8 for CVE-2026-54512 and CVE-2026-54513 #4765 @sulikismaylovv Updates bundled Jackson dependencies across Java, Spark, Flink, and Hive and adjusts shaded jars for the newer release.
• Java integrations: Upgrade Jackson to 2.18.9 #4853 @Poojitha-R-Rao Applies the subsequent Jackson patch release consistently across the Java client and integrations.
• Spark: Apply removePath.pattern to RDD datasets #4726 @zerafachris Applies configured dataset path removal to RDD job inputs and outputs, matching SQL and DataFrame behavior.
• Spark: Clean up retained lifecycle state #4894 @mobuchowski Evicts completed jobs, stages, and metrics so long-lived Spark drivers do not retain state without bound.
• Spark: Extract Delta output from V1 micro-batch writes #4884 @MSDehghan Restores output datasets for Spark 4 structured-streaming writes through Delta's V1 sink.
• Spark: Fix Databricks COPY INTO lineage #4850 @mishrasangeeta87 Emits source inputs and Unity Catalog target outputs for proprietary Databricks COPY INTO plan variants.
• Spark: Fix Databricks CTAS output lineage #4849 @mishrasangeeta87 Restores output datasets for CTAS and related V2 create or replace commands on Databricks runtimes.
• Spark: Fix Databricks DELETE lineage #4815 @mishrasangeeta87 Emits the DELETE target as an output and tables referenced by predicate subqueries as inputs.
• Spark: Fix Databricks UPDATE lineage #4835 @mishrasangeeta87 Emits UPDATE targets and SET or WHERE subquery inputs from proprietary Databricks plan nodes.
• Spark: Prevent read-only V2 plans from becoming outputs #4898 @MSDehghan Stops read-only V2 scans, such as lazy Iceberg checkpoints, from being reported as writes.
• Spark: Register the V1 micro-batch output builder for Spark 4 #4772 @MSDehghan Restores output lineage for Spark 4 structured-streaming jobs that use V1 sinks.
• Spark: Report Iceberg writes through the cached catalog #4896 @JDarDagran Resolves maintenance-action datasets from the underlying Iceberg relation when writes use SparkCachedTableCatalog.
• Spark: Support Spark 4.2 CatalogManager compatibility #4779 @mobuchowski Avoids listener failures after Spark 4.2 changed CatalogManager from a class to an interface.
• SQL: Fix MySQL multi-table DELETE lineage #4775 @mattfaltyn Classifies explicit DELETE targets as outputs and joined lookup tables as inputs.
• SQL: Fix Snowflake COPY unload lineage #4791 @mattfaltyn Reports source tables as inputs and the destination location as output for Snowflake unload statements.
• SQL: Include aggregate filter lineage #4865 @mattfaltyn Captures predicate columns and subquery tables referenced only by aggregate FILTER expressions.
• SQL: Include ARRAY subquery inputs #4763 @mattfaltyn Includes tables referenced by BigQuery ARRAY subqueries in input lineage.
• SQL: Include HAVING subquery inputs #4785 @mattfaltyn Includes tables read only by subqueries in HAVING expressions.
• SQL: Include JOIN-condition subquery inputs #4783 @mattfaltyn Includes tables read only by subqueries in JOIN ... ON conditions.
• SQL: Include MERGE-predicate subquery inputs #4855 @mattfaltyn Includes tables referenced only by subqueries in MERGE ON predicates.
• SQL: Include UPDATE-assignment subquery inputs #4767 @mattfaltyn Includes tables read by subqueries in UPDATE SET assignment expressions.
• SQL: Include VALUES subquery inputs #4788 @mattfaltyn Includes tables read by scalar subqueries inside VALUES rows.
• SQL: Preserve lineage through derived PIVOT inputs #4852 @mattfaltyn Retains source datasets when a Snowflake PIVOT wraps a derived table.