Python: add CompositeTransport#2925@JDarDagran Adds a CompositeTransport that can accept other transport configs to instantiate transports and use them to emit events.
Spark: compile & test Spark integration on Java 17#2828@pawel-big-lebowski The Spark integration is always compiled with Java 17, while tests are running on both Java 8 and Java 17 according to the configuration.
Spark: support preview release of Spark 4.0#2854@pawel-big-lebowski Includes the Spark 4.0 preview release in the integration tests.
Spark: add handling for Window#2901@tnazarew Adds handling for Window-type nodes of a logical plan.
Spark: extract and send events with raw SQL from Spark#2913@Imbruced Adds a parser that traverses QueryExecution to get the SQL query used from the SQL field with a BFS algorithm.
Spark: support Mongostream source#2887@Imbruced Adds a Mongo streaming visitor and tests.
Spark: new mechanism for disabling facets#2912@arturowczarek The mechanism makes FacetConfig accept the disabled flag for any facet instead of passing them as a list.
Spark: support Kinesis source#2906@Imbruced Adds a Kinesis class handler in the streaming source builder.
Spark: extract DatasetIdentifier from extension LineageNode#2900@ddebowczyk92 Adds support for cases in which LogicalRelation has a grandChild node that implements the LineageRelation interface.
Spark: extract Dataset from underlying BaseRelation#2893@ddebowczyk92 DatasetIdentifier is now extracted from the underlying node of LogicalRelation.
Spark: add descriptions and Marquez UI to Docker Compose file#2889@jonathanlbt1 Adds the marquez-web service to docker-compose.yml.
Proxy: bug fixed on error messages descriptions#2880@jonathanlbt1 Improves error logging.
Proxy: update Docker image for Fluentd 1.17#2877@jonathanlbt1 Upgrades the Fluentd version.
Spark: fix issue with Kafka source when saving with for each batch method#2868@imbruced Fixes an issue when Spark is in streaming mode and input for Kafka was not present in the event.
Spark: properly set ARN in namespace for Iceberg Glue symlinks#2943@arturowczarek Makes IcebergHandler support Glue catalog tables and create the symlink using the code from PathUtils.
Spark: accept any provider for AWS Glue storage format#2917@arturowczarek Makes the AWS Glue ARN generating method accept every format (including Parquet), not only Hive SerDe.
Spark: return valid JSON for failed logical plan serialization#2892@arturowczarek The LogicalPlanSerializer now returns <failed-to-serialize-logical-plan> for failed serialization instead of an empty string.
Spark: add Kafka input source when writing in foreach batch mode#2868@Imbruced Fixes a bug keeping Kafka input sources from being produced.
Spark: extract DatasetIdentifier from SaveIntoDataSourceCommandVisitor options#2934@ddebowczyk92 Extracts DatasetIdentifier from command's options instead of relying on p.createRelation(sqlContext, command.options()), which is a heavy operation for JdbcRelationProvider.