Java: enable specifying custom SSL context#3444@pawel-big-lebowskiEnable providing configuration for SSL context within HTTP transport.
Spark: make Spark nodes filtering configurable.#3442@pawel-big-lebowskiSpark integration filters OpenLineage events for specific plan node classes. This can be now extended with extra config entries: allowedSparkNodes and deniedSparkNodes. See Spark Configuration documentation for more details.
Java: add task queue based async circuit breaker.#3437@aritrabandyoThis circuit breaker that executes task on a queue backed threadpool, gives up tasks if the queue is full, and keeps track of rejected tasks.
dbt: added initial support for Trino adapter#3429@whitleykeithThis allows Trino integration to emit proper events containing Trino datasets.
Spark: increased coverage for Spark DML commands#3430@ssanthanam185Adds coverage for AlterTableRecoverPartitionsCommandVisitor, RefreshTableCommandVisitor, RepairTableCommandVisitor.
Spark: the OpenLineageSparkListener was refactored to have a public, single-argument constructor taking an instance of SparkConf.#3425@d-m-hThis presents no functional change to the listener, however it will allow for improved initialisation of the listener in the future.
Spark: Unsupported catalog exception should be less verbose.#3435@pawel-big-lebowskiIn case of unsupported classes, warn logs without a stacktrace should be produced.
Spark: Directly expose the LogicalPlan and SparkPlan objects inside OpenLineageContext.#3443@d-m-hThis is an initial refactor to a larger code base change that will see the removal of direct access of the QueryExecution object. It has no functional change on the way the integration behaves.
Spark: improve column lineage by including inputs within COMPLETE events.#3434@pawel-big-lebowski
*Send input datasets in COMPLETE events while making sure version facet is attached on START only.
dbt: ParentRunFacet is now correctly attached when using structured logs option.#3432@MassyBFixes incorrect structure of ParentRunFacet.