Skip to main content
Version: Next

1.21.1 - 2024-08-29

Added​

  • Spec: add GCP Dataproc facet #2987 @tnazarew
    Registers the Google Cloud Platform Dataproc run facet.

Fixed​

  • Airflow: update SQL integration code to work with latest sqlparser-rs main #2983 @kacpermuda
    Adjusts the SQL integration after our sqlparser-rs fork has been updated to the latest main.
  • Spark: fix AWS Glue jobs naming for SQL events #3001 @arturowczarek
    SQL events now properly use the names of the jobs retrieved from AWS Glue.
  • Spark: fix issue with column lineage when using delta merge into command #2986 @Imbruced
    A view instance of a node is now included when gathering data sources for input columns.
  • Spark: minor Spark filters refactor #2990 @arturowczarek
    Fixes a number of minor issues.
  • Spark: Iceberg tables in AWS Glue have slashes instead of dots in symlinks #2984 @arturowczarek
    They should use slashes and the prefix table/.
  • Spark: lineage for Iceberg datasets that are present outside of Spark's catalog is now present #2937 @d-m-hPreviously, reading Iceberg datasets outside the configured Spark catalog prevented the datasets from being present in the inputs property of the RunEvent.