Skip to main content

Integrating With OpenLineage

info

This page has not yet been written! You're welcome to contribute using the Edit link at the bottom.

Capability Matrix

caution

This matrix is not yet complete.

The matrix below shows the relationship between an input facet and various mechanisms OpenLineage uses to gather metadata. Not all mechanisms collect data to fill in all facets, and some facets are specific to one integration. ✔️: The mechanism does implement this facet. ✖️: The mechanism does not implement this facet. An empty column means it is not yet documented if the mechanism implements this facet.

MechanismIntegrationMetadata GatheredInputDatasetFacetOutputDatasetFacetSqlJobFacetSchemaDatasetFacetDataSourceDatasetFacetDataQualityMetricsInputDatasetFacetDataQualityAssertionsDatasetFacetSourceCodeJobFacetExternalQueryRunFacetDocumentationDatasetFacetSourceCodeLocationJobFacetDocumentationJobFacetParentRunFacet
SnowflakeOperator*Airflow ExtractorLineage
Job duration
✔️✔️✔️✔️✔️✖️✖️
BigQueryOperator**Airflow ExtractorLineage
Schema details
Job duration
✔️✔️✔️
PostgresOperator*Airflow ExtractorLineage
Job duration
✔️✔️✔️✔️✔️
SqlCheckOperatorsAirflow ExtractorLineage
Data quality assertions
✔️✖️✔️✔️✔️✔️✔️
dbtdbt Project FilesLineage
Row count
Byte count.
✔️
Great ExpectationsActionData quality assertions✔️✔️✔️
SparkSparkListenerSchema
Row count
Column lineage
✔️
Snowflake***Access HistoryLineage
  • Uses the Rest SQL parser Uses the BigQuery API * Uses Snowflake query logs

Integration strategies

Integrating with pipelines

Integrating with Pipelines

Integrating with data sources

Integrating with Data Sources