1.24.2 - 2024-11-05
Added
- Spark: Add Dataproc run facet to include jobType property
#3167@codelixir
Updates the GCP Dataproc run facet to include jobType property. - Add EnvironmentVariablesRunFacet to core spec
#3186@JDarDagran
Additionally, directly use EnvironmentVariablesRunFacet in Python client. - Add assertions for format in test events
#3221@JDarDagran - Spark: Add integration tests for EMR
#3142@arturowczarek
Spark integration has integration tests for EMR.
Changed
- Move Kinesis to separate module, migrate HTTP transport to httpclient5
#3205@mobuchowski
Moves Kinesis integration to a separate module and updates HTTP transport to use HttpClient 5.x. - Docs: Upgrade docusaurus to 3.6
#3219@arturowczarek - Spark: Limit the Seq size in RddPathUtils::extract()
#3148@codelixir
Adds flag to limit the logs in RddPathUtils::extract() to avoid OutOfMemoryError for large jobs.
Fixed
- Docs: Fix outdated Spark-related docs
#3215@mobuchowski - Fix docusaurus-mdx-checker errors
#3217@arturowczarek - [Integration/dbt] Parse dbt source tests
#3208@MassyB
Consider dbt sources when looking for test results. - Avoid tests in configurable test
#3141@pawel-leszczynski