Trino Integration with OpenLineage

ADOC integrates with Trino using OpenLineage to capture query execution metadata and automatically build pipeline observability. When configured, Trino emits OpenLineage events for every qualifying query. ADOC ingests these events to construct pipelines and pipeline runs, providing visibility into how data is processed and moved across Trino workloads.

How Trino Integration with OpenLineage Works

• Trino emits OpenLineage events for each qualifying query execution.
• ADOC ingests these events and extracts query metadata.
• Each unique query pattern is modeled as a Pipeline.
• Each execution of that query is modeled as a Pipeline Run.

Example: Consider the following queries executed on a Trino table T1:

xxxxxxxxxx

INSERT INTO T1 VALUES (1); 

INSERT INTO T1 VALUES (2);

In ADOC, this results in:

• 1 Pipeline → INSERT INTO T1
• 2 Pipeline Runs → one per execution

This keeps pipelines stable while capturing execution-level observability.

Pipeline Stitching at Query Level

Unlike traditional job-based pipelines, Trino pipelines in ADOC are stitched at the query level.

• The query signature defines the pipeline.
• Each runtime execution of that query becomes a pipeline run.
• Inputs and outputs are automatically derived from OpenLineage metadata.

This approach allows ADOC to model highly dynamic, ad-hoc Trino workloads without requiring predefined pipeline definitions.

Enhanced Lineage and Related Pipelines

With this integration, ADOC can:

• Identify pipelines reading from a table
• Identify pipelines writing to a table
• Show upstream and downstream dependencies

Use cases:

• Impact analysis
• Data quality investigations
• Change management

Configuration

Complete the following steps on your Trino server to enable OpenLineage event emission to ADOC.

Step 1: Install the OpenLineage Plugin

The OpenLineage plugin must be present in the Trino plugin directory before the event listener can be configured.

1. Navigate to the Trino plugin directory:

xxxxxxxxxx

cd /opt/trino/server/plugin sudo mkdir -p openlineage cd openlineage

2. Download the plugin JAR:

xxxxxxxxxx

sudo wget https://repo1.maven.org/maven2/io/openlineage/openlineage-trino/1.23.0/openlineage-trino-1.23.0.jar

3. Verify the download:

xxxxxxxxxx

ls -lh

Expected output: openlineage-trino-1.23.0.jar

Step 2: Configure the OpenLineage Event Listener

Create or update the event listener configuration file at /etc/trino/event-listener.properties with the following:

event-listener.name=openlineage

# Trino server URI 

openlineage-event-listener.trino.uri=http://<TRINO_HOST>:8080

# Transport configuration

openlineage-event-listener.transport.type=http openlineage-event-listener.transport.url=https://<TENANT_NAME>.acceldata.app/torch-pipeline/api/v1/lineage

# Authentication headers 

openlineage-event-listener.transport.headers=accessKey:<ADOC_ACCESS_KEY>,secretKey:<ADOC_SECRET_KEY>;

# Query types to capture

openlineage-event-listener.trino.include-query-types=INSERT,CREATE_TABLE_AS_SELECT,MERGE,DELETE,UPDATE

# Namespace for lineage events

openlineage-event-listener.namespace=trino-production

Replace the placeholders with values from your environment:

Step 3: Configure TLS TrustStore

4. Download the certificate from the ADOC UI and save it as acceldata-cert.pem.
5. Place the file in a directory, for example certs.
6. Generate a truststore using the following command:

xxxxxxxxxx

keytool -import -trustcacerts -noprompt \ 

  -alias acceldata \ 

  -file ./acceldata-cert.pem \ 

  -keystore ./truststore.jks \ 

  -storepass changeit

7. Mount the certs directory in the Trino instance at /etc/trino/certs.

Step 4: Restart Trino

Restart the Trino service to apply the configuration changes.

xxxxxxxxxx

sudo systemctl restart trino 

sudo systemctl status trino

Note If you are not running Trino as a systemd service, stop and restart the server manually using ./bin/launcher stop followed by ./bin/launcher start from your Trino installation directory.

Step 5: Verify the Integration

After Trino restarts, confirm the OpenLineage listener has loaded correctly.

Check the server log

Run the following command to confirm the event listener is registered:

xxxxxxxxxx

grep -i "openlineage" /var/trino/data/var/log/server.log

Expected log entry: Registered event listener openlineage

Confirm Trino is listening

Verify that Trino is accepting connections on port 8080:

xxxxxxxxxx

sudo netstat -tlnp | grep 8080

A successful response shows port 8080 in a listening state.

Confirm events are reaching ADOC

Execute a qualifying query on Trino, then navigate to the Pipelines view in ADOC to confirm a pipeline and pipeline run have been created for the query.

What's Next

After completing this configuration, explore the following topics:

• Pipelines – Understand how ADOC models Trino queries as pipelines and pipeline runs.
• Lineage Explorer – Trace upstream and downstream data dependencies across Trino workloads.
• Impact Analysis – Assess the downstream effect of changes to Trino tables or query patterns.
• Data Reliability – Apply reliability policies to tables written to by Trino pipelines.