1.3.1 - 2023-10-03
Added
- Airflow: add some basic stats to the Airflow integration
#1845
@harels
Uses the statsd component that already exists in the Airflow codebase and wraps the section that emits to event with a timer, as well as emitting a counter for exceptions in sending the event. - Airflow: add columns as schema facet for
airflow.lineage.Table
(if defined)#2138
@erikalfthan
Adds columns (if set) fromairflow.lineage.Table
inlets/outlets to the OpenLineage Dataset. - DBT: add SQLSERVER to supported dbt profile types
#2136
@erikalfthan
Adds support for dbt-sqlserver, solving #2129. - Spark: support for latest 3.5
#2118
@pawel-big-lebowski
Integration tests are now run on Spark 3.5. Also upgrades 3.3 branch to 3.3.3. Please note thatdelta
andiceberg
are not supported for Spark3.5
at this time.
Fixed
- Airflow: fix find-links path in tox
#2139
@JDarDagran
Fixes a broken link. - Airflow: add more graceful logging when no OpenLineage provider installed
#2141
@JDarDagran
Recognizes a failed import ofairflow.providers.openlineage
and adds more graceful logging to fix a corner case. - Spark: fix bug in PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) to correctly preserve scheme from CatalogTable's location
#2142
@d-m-h
Previously, theprepareDatasetIdentifierFromDefaultTablePath
method would override the scheme with the value of "file" when constructing a dataset identifier. It now uses the scheme of theCatalogTable
's URI for this. Thank you @pawel-big-lebowski for the quick triage and suggested fix.