1.6.2 - 2023-12-07
Added
- Dagster: support Dagster 1.5.x
#2220@tsungchih
Gets event records for each target Dagster event type to support Dagster version 0.15.0+. - Dbt: add a new command
dbt-ol send-eventsto send metadata of the last run without running the job#2285@sophiely
Adds a new command to send events to OpenLineage according to the latest metadata generated without running any dbt command. - Flink: add option for Flink job listener to read from Flink conf
#2229@ensctom
Adds option for the Flink job listener to read jobnames and namespaces from Flink conf. - Spark: get column-level lineage from JDBC dbtable option
#2284@mobuchowski
Adds support for dbtable, enables lineage in the case of single input columns, and improves dataset naming. - Spec: introduce
JobTypeJobFacetto contain additional job related information#2241@pawel-big-lebowski
NewJobTypeJobFacetcontains the processing type such asBATCH|STREAMING, integration viaSPARK|FLINK|...and job type inQUERY|COMMAND|DAG|.... - SQL: add quote information from sqlparser-rs
#2259@JDarDagran
Adds quote information from sqlparser-rs.
Fixed
- Spark: update Jackson dependency to resolve
CVE-2022-1471#2185@pawel-big-lebowski
Updates Gradle for Spark and Flink to 8.1.1. Upgrade Jackson2.15.3. - Flink: avoid relying on Guava which can be missing during production runtime
#2296@pawel-big-lebowski
Removes usage of Guava ImmutableList. - Spark: exclude
commons-loggingtransitive dependency from published jar#2297@pawel-big-lebowski
Ensurescommons-loggingis not shipped as this can lead to a version mismatch on the user's side.