1.12.0 - 2024-04-09
Added
- Airflow: add
lineage_job_namespace
andlineage_job_name
macros#2582
@dolfinus
Adds new Airflow macroslineage_job_namespace()
,lineage_job_name(task)
that return an Airflow namespace and Airflow job name, respectively. - Spec: Allow nested struct fields in
SchemaDatasetFacet
#2548
@dolfinus
Allows nested fields support toSchemaDatasetFacet
.
Fixed
- Spark: fix PMD for test
#2588
@pawel-big-lebowski
ClearspmdTestScala212
from warnings that clutter the logs. - Dbt: propagate the dbt return code also when no OpenLineage events are emitted
#2591
@blacklight
dbt-ol
now propagates the exit code of the underlying dbt process even if no lineage events are emitted. - Java: make sure string isn't empty to prevent going out of bounds
#2585
@harels
String lookup was not accounting for empty strings and causing ajava.lang.StringIndexOutOfBoundsException
. - Spark: use
HashSet
in column-level lineage instead of iterating throughLinkedList
#2584
@mobuchowski
Takes advantage of performance gains available from usingHashSet
for collection. - Python: fix missing pkg_resources module on Python 3.12
#2572
@dolfinus
Removespkg_resources
dependency and replaces it with the packaging lib. - Airflow: fix format returned by
airflow.macros.lineage_parent_id
#2578
@blacklight
Fixes the run format returned by thelineage_parent_id
Airflow macro and simplifies the format of thelineage_parent_id
andlineage_run_id
macros. - Dagster: limit Dagster version to 1.6.9
#2579
@JDarDagran
Adds an upper limit on supported versions of Dagster as the integration is no longer actively maintained and recent releases introduce breaking changes.