Skip to main content
Version: 1.42.1

Setup a development environment

There are four Python OpenLineage packages that you can install locally when setting up a development environment:
openlineage-python (client), openlineage-sql and openlineage-integration-common.

The repository uses UV for Python dependency management with path-based dependencies, where each integration is a standalone project with isolated dependencies.

Prerequisites​

Install UV if you haven't already:

$ curl -LsSf https://astral.sh/uv/install.sh | sh

Quick Start with Makefile​

The repository includes a Makefile to simplify the development environment setup:

# View all available commands
$ make help

# Setup all Python integrations at once
$ make setup-all

# Or setup specific integrations
$ make setup-client      # Python client
$ make setup-common      # Integration common library
$ make setup-dbt         # dbt integration

# Run tests
$ make test-all          # Test all integrations
$ make test-client       # Test specific integration

# Run linting and type checking
$ make lint-all          # Run all linting
$ make fix-format        # Auto-fix formatting issues

# Check status of your setup
$ make status

# Clean all virtual environments
$ make clean

Manual Setup​

If you prefer to set up integrations manually:

# Python client
$ cd client/python
$ uv sync --extra dev --extra test

# Integration common
$ cd integration/common
$ uv sync --extra dev

# dbt integration
$ cd integration/dbt
$ uv sync --extra dev

How Path-Based Dependencies Work​

The repository uses path-based dependencies instead of a UV workspace because each integration has potentially conflicting dependencies. Each integration is a standalone project with its own isolated virtual environment.

Each integration automatically installs its dependencies from local directories in editable mode:

  • dbt integration depends on common package
  • Common integration depends on client and sql packages

UV handles these path-based dependencies automatically, so changes in one package are immediately reflected in dependent packages without reinstallation.

Testing​

There are unit tests available for OpenLineage Python libraries. You can run them with a simple pytest command with directory set to library base path.