1.12.0 - 2024-04-09
Added
- Airflow: add
lineage_job_namespaceandlineage_job_namemacros#2582@dolfinus
Adds new Airflow macroslineage_job_namespace(),lineage_job_name(task)that return an Airflow namespace and Airflow job name, respectively. - Spec: Allow nested struct fields in
SchemaDatasetFacet#2548@dolfinus
Allows nested fields support toSchemaDatasetFacet.
Fixed
- Spark: fix PMD for test
#2588@pawel-big-lebowski
ClearspmdTestScala212from warnings that clutter the logs. - Dbt: propagate the dbt return code also when no OpenLineage events are emitted
#2591@blacklight
dbt-olnow propagates the exit code of the underlying dbt process even if no lineage events are emitted. - Java: make sure string isn't empty to prevent going out of bounds
#2585@harels
String lookup was not accounting for empty strings and causing ajava.lang.StringIndexOutOfBoundsException. - Spark: use
HashSetin column-level lineage instead of iterating throughLinkedList#2584@mobuchowski
Takes advantage of performance gains available from usingHashSetfor collection. - Python: fix missing pkg_resources module on Python 3.12
#2572@dolfinus
Removespkg_resourcesdependency and replaces it with the packaging lib. - Airflow: fix format returned by
airflow.macros.lineage_parent_id#2578@blacklight
Fixes the run format returned by thelineage_parent_idAirflow macro and simplifies the format of thelineage_parent_idandlineage_run_idmacros. - Dagster: limit Dagster version to 1.6.9
#2579@JDarDagran
Adds an upper limit on supported versions of Dagster as the integration is no longer actively maintained and recent releases introduce breaking changes.