Skip to main content
Version: 1.23.0

1.12.0 - 2024-04-09

Added

  • Airflow: add lineage_job_namespace and lineage_job_name macros #2582 @dolfinus
    Adds new Airflow macros lineage_job_namespace(), lineage_job_name(task) that return an Airflow namespace and Airflow job name, respectively.
  • Spec: Allow nested struct fields in SchemaDatasetFacet #2548 @dolfinus
    Allows nested fields support to SchemaDatasetFacet.

Fixed

  • Spark: fix PMD for test #2588 @pawel-big-lebowski
    Clears pmdTestScala212 from warnings that clutter the logs.
  • Dbt: propagate the dbt return code also when no OpenLineage events are emitted #2591 @blacklight
    dbt-ol now propagates the exit code of the underlying dbt process even if no lineage events are emitted.
  • Java: make sure string isn't empty to prevent going out of bounds #2585 @harels
    String lookup was not accounting for empty strings and causing a java.lang.StringIndexOutOfBoundsException.
  • Spark: use HashSet in column-level lineage instead of iterating through LinkedList #2584 @mobuchowski
    Takes advantage of performance gains available from using HashSet for collection.
  • Python: fix missing pkg_resources module on Python 3.12 #2572 @dolfinus
    Removes pkg_resources dependency and replaces it with the packaging lib.
  • Airflow: fix format returned by airflow.macros.lineage_parent_id #2578 @blacklight
    Fixes the run format returned by the lineage_parent_id Airflow macro and simplifies the format of the lineage_parent_id and lineage_run_id macros.
  • Dagster: limit Dagster version to 1.6.9 #2579 @JDarDagran
    Adds an upper limit on supported versions of Dagster as the integration is no longer actively maintained and recent releases introduce breaking changes.