1.36.0 - 2025-07-22

Added

• Spark: support Delta 4.0 and cover it with tests on Spark 4.0. #3877 @pawel-big-lebowski
  Fix failing tests for Spark 4.0. Make delta integration tests pass with Delta 4.0 on Spark 4.
• Spark: Add memory info to debug facet. #3914 @pawel-big-lebowski
  Extend DebugFacet with additional information on Spark's driver memory configuration and current memory usage.
• Spark: Add new AlterTableCommandDatasetBuilder for Spark 4.0. #3921 @pawel-big-lebowski
  Add support for AlterTableCommand dataset building in Spark 4.0.
• dbt: Add query IDs for dbt. #3890 @jroachgolf84
  Add query ID tracking to dbt integration.
• dbt: Add query ID capture in structured logs. #3918 @mobuchowski
  Capture query IDs from dbt structured logs for better traceability.
• Python: Formalize dataset naming for Python client. #3816 @ddebowczyk92
  Formalize dataset naming conventions in Python client implementation.

Changed

• Spark: bump minor versions 3.4.3 -> 3.4.4, 3.5.4 -> 3.5.6. #3907 @pawel-big-lebowski
  Bump tested Spark versions.
• Spark: Close OpenLineageClient in onApplicationEnd. #3851 @dolfinus
  Ensure proper cleanup of OpenLineageClient when Spark application ends.
• Python: Do not use f-strings with logging module. #3895 @mobuchowski
  Replace f-string usage in logging calls with proper logging formatting.
• Python: Update protobuf version to be compatible with newer libraries. #3899 @mobuchowski
  Update protobuf dependency to maintain compatibility with newer library versions.
• Website: Documentation for compatibility tests. #3869 @mobuchowski
  Add documentation explaining compatibility testing processes.

Fixed

• Spark: make visitors stateless - avoid memory leak. #3902 @pawel-big-lebowski
  Merge SqlExecutionRDDVisitor and LogicalRDDVisitor classes to avoid memory leak.
• Spark: refactor iceberg handler. #3909 @pawel-big-lebowski
  Refactor Iceberg handler implementation for better maintainability.
• Spark: retry exception on empty row. #3908 @pawel-big-lebowski
  Add retry logic for handling empty row exceptions.
• Spark: fix Spark version for databricks test. #3911 @pawel-big-lebowski
  Fix Spark version configuration in Databricks test environment.
• Flink: Fix connector of type kafka-upsert not identifying kafka topics correctly. #3915 @fetta
  Fix kafka-upsert connector to properly identify kafka topics.
• Airflow: Fail fast and reduce timeout for airflow tests. #3905 @kacpermuda
  Improve test performance by implementing fail-fast behavior and reduced timeouts.
• dbt: more telemetry, fix quadratic file reading. #3916 @mobuchowski
  Improve telemetry collection and fix performance issues with file reading.
• dbt: Fix dbt version. #3894 @mobuchowski
  Fix dbt version compatibility issues.
• Python: Fix filenames for windows users. #3889 @pawel-big-lebowski
  Fix filename handling to work correctly on Windows systems.
• Transport: Adjust log level when aliasing default_http transport. #3897 @dolfinus
  Adjust logging level for transport aliasing messages.
• Build: Improve comments and add some tests. #3901 @kacpermuda
  Improve code documentation and add additional test coverage.