Skip to main content

OpenLineage Integrations

Capability Matrix


This matrix is not yet complete.

The matrix below shows the relationship between an input facet and various mechanisms OpenLineage uses to gather metadata. Not all mechanisms collect data to fill in all facets, and some facets are specific to one integration.
✔️: The mechanism does implement this facet.
✖️: The mechanism does not implement this facet.
An empty column means it is not yet documented if the mechanism implements this facet.

MechanismIntegrationMetadata GatheredInputDatasetFacetOutputDatasetFacetSqlJobFacetSchemaDatasetFacetDataSourceDatasetFacetDataQualityMetricsInputDatasetFacetDataQualityAssertionsDatasetFacetSourceCodeJobFacetExternalQueryRunFacetDocumentationDatasetFacetSourceCodeLocationJobFacetDocumentationJobFacetParentRunFacet
SnowflakeOperator*Airflow ExtractorLineage
Job duration
BigQueryOperator**Airflow ExtractorLineage
Schema details
Job duration
PostgresOperator*Airflow ExtractorLineage
Job duration
SqlCheckOperatorsAirflow ExtractorLineage
Data quality assertions
dbtdbt Project FilesLineage
Row count
Byte count.
Great ExpectationsActionData quality assertions✔️✔️✔️
Row count
Column lineage
Snowflake***Access HistoryLineage

* Uses the Rest SQL parser
** Uses the BigQuery API
*** Uses Snowflake query logs

Compatibility matrix

This matrix shows which data sources are known to work with each integration, along with the minimum versions required in the target system or framework.

PlatformVersionData Sources
Apache Airflow1.10+
Amazon Athena
Amazon Redshift
Amazon SageMaker
Amazon S3 Copy and Transform
Google BigQuery
Google Cloud Storage
Great Expectations
Apache Spark2.4+JDBC
Google Cloud Storage
Google BigQuery
Amazon S3
Azure Blob Storage
Azure Data Lake Gen2
Azure Synapse
Google BigQuery

Integration strategies


This section could use some more detail! You're welcome to contribute using the Edit link at the bottom.

Integrating with pipelines

Integrating with Pipelines

Integrating with data sources

Integrating with Data Sources