0.20.4 - 2023-02-07
- Airflow: add new extractor for
Adds a new extractor for this operator.
- Flink: resolve topic names from regex, support 1.16.0
Adds support for Flink 1.16.0 and makes the integration resolve topic names from Kafka topic patterns.
- Proxy: implement lineage event validator for client proxy
Implements logic in the proxy (which is still in development) for validating and handling lineage events.
- CI: use
ruffinstead of flake8, isort, etc., for linting and formatting
ruffpackage, which combines several linters and formatters into one fast binary.
- Airflow: make the Trino catalog non-mandatory
Makes the Trino catalog optional in the Trino extractor.
- Common: add explicit SQL dependency
Addresses a 0.19.2 breaking change to the GE integration by including the SQL dependency explicitly.
- DBT: adjust
tqdmto show the correct number of iterations and adds START events for parent runs.
- DBT: fix typo in log output
Fixes 'emittled' typo in log output.
- Great Expectations/Airflow: follow Snowflake dataset naming rules
Normalizes Snowflake dataset and datasource naming rules among DBT/Airflow/GE; canonizes old Snowflake account paths around making them all full-size with account, region and cloud names.
- Java and Python Clients: Kafka does not initialize properties if they are empty; check and notify about Confluent-Kafka requirement
Fixes the failure to initialize
KafkaTransportin the Java client and adds an exception if the required
confluent-kafkamodule is missing from the Python client.
- Spark: add square brackets for list-based Spark configs
Adds a condition to treat configs with
as lists. Note:
will be required for list-based configs starting with 0.21.0.
- Spark: fix several Spark/BigQuery-related issues
Fixes the assumption that a version is always a number; adds support for
HadoopMapReduceWriteConfigUtil; makes the integration access
getTableIdusing reflection, which supports all BigQuery versions; makes logs provide the full serialized LogicalPlan on
- SQL: only report partial failures `#1479 @mobuchowski
Changes the parser so it reports partial failures instead of failing the whole extraction.