This page shows you how to get the tables and other data your agent reads and writes onto its spans, so AIO can match them to assets in the Acceldata (ADOC) catalog.
AIO gets data references from two places:
- Detected references come from the SQL that a database instrumentation records on a span.
- Declared references come from your code, through
declare_assets. Use them for data the SQL can't show, such as files and endpoints.
To learn how AIO matches references to the catalog and shows their data quality, see Data quality overview.
Prerequisites
- The AIO Python SDK is installed and sends traces. For more information, see Instrument your code.
- The data sources your agent uses are registered in the ADOC catalog. A reference that doesn't match a registered data source shows as not in the catalog.
Capture SQL with a database instrumentation
When your agent queries a database through an instrumented driver, the span records the SQL statement. AIO reads each table out of the statement and records one detected reference per table.
- Enable the database instrumentation for your driver through the AIO Python SDK.
- Run your agent so it sends a trace that queries the database.
- Open the trace in AIO and check the Assets used panel. For more information, see Explore traces.
AIO waits 60 seconds for late spans before it reads the SQL, so detected references show up shortly after the trace arrives.
Supported drivers
The SDK captures SQL from the following drivers:
psycopg(version 3) andpsycopg-pool- psycopg2
- asyncpg
- aiopg
- pymysql
- mysql (
mysql.connector) - mysqlclient (
MySQLdb) - pymssql
- BigQuery (
Client.queryandquery_and_wait) - Databricks (
databricks.sql) - Snowflake (
snowflake.connector) - Trino (
trino.dbapi) - Athena (
pyathena)
SQLAlchemy is covered through whichever of these drivers it uses.
How detected references are named
Each detected reference has a namespace, built from the database system and the connection's host and port, and a name. AIO qualifies table names with the connection's database and the system's default schema. For example, a query on customers in the orders Postgres database on orders-db.acme.com becomes namespace postgres://orders-db.acme.com:5432 and name orders.public.customers.
The following table shows how each supported system is named.
| System | Namespace | Default port | Default schema |
|---|---|---|---|
| PostgreSQL | postgres:// and the host |
5432 | public |
Redshift (host contains .redshift. or .redshift-serverless.) |
— | 5439 | — |
| Snowflake | snowflake://<account>, with .snowflakecomputing.com removed |
— | — |
| Databricks | The host only | — | — |
| BigQuery | bigquery |
— | — |
| Trino | — | — | — |
| Athena | awsathena:// |
— | — |
| MySQL | — | 3306 | — |
| SQL Server | — | 1433 | dbo |
AIO also does the following when it reads a statement:
- Leaves out system schemas and tables, such as
pg_catalogand objects whose names start withpg_. - Skips common table expressions (CTEs) and derived tables.
- Uppercases unquoted Snowflake names.
- Records how each column was used:
select,filter(inWHEREorHAVING),join,group,order, orwrite(inSETor an insert column list).
Send SQL from your own instrumentation
AIO treats any span that has db.query.text or db.statement as a SQL span, so spans from other OpenTelemetry instrumentations work too. AIO reads the connection details from these attributes, in order of preference:
| Detail | Attributes |
|---|---|
| System | db.system.name, then db.system |
| Host and port | aio.data.source.address, then server.address, then net.peer.name |
| Database name | db.namespace, then db.name |
Declare data the SQL can't show
Call aio.declare_assets in code that runs inside a span to record what that code reads and writes. Use it for files, endpoints, collections, or tables your agent reaches without an instrumented driver.
declare_assets takes keyword-only reads and writes. Each item is either a (namespace, name) tuple or a dictionary with these keys:
| Key | Required | Value |
|---|---|---|
namespace |
Yes | Where the data lives, for example postgres://orders-db.acme.com:5432 |
name |
Yes | The data's name, for example orders.public.customers |
kind |
No | table, view, file, endpoint, or collection |
columns |
No | A list or other iterable of column names. A bare string is ignored. |
import acceldata_aio_tracer as aio
def load_customer_orders(customer_id):
# Runs inside a span, for example a tool call your agent makes.
aio.declare_assets(
reads=[
("postgres://orders-db.acme.com:5432", "orders.public.customers"),
{
"namespace": "postgres://orders-db.acme.com:5432",
"name": "orders.public.orders",
"kind": "table",
"columns": ["customer_id", "order_date", "total"],
},
],
writes=[
("postgres://orders-db.acme.com:5432", "orders.public.order_summaries"),
],
)
Here's how declare_assets behaves:
- It adds the references to the current span. Declared references show as Declared in the UI.
- Calling it again on the same span adds to the list. A reference you declare twice is recorded once.
- Columns are sorted, and duplicates are removed.
- It does nothing when no span is open or the span isn't recording.
- It never raises an error to your code. If a reference is invalid, for example because it's missing a namespace or a name, the SDK logs the warning
aio: could not record declared assets.
Map connection hosts to catalog hosts
When your application connects through localhost or a tunnel, the host on the span doesn't match the host registered in the catalog, so the reference doesn't match. Map the local host to the registered one.
Pass data_source_hosts to aio.init:
aio.init(
data_source_hosts={"localhost:5432": "orders-db.acme.com:5432"},
)
Or set the AIO_DATA_SOURCE_HOSTS environment variable, in the form from=to,from=to:
export AIO_DATA_SOURCE_HOSTS="localhost:5432=orders-db.acme.com:5432,localhost:3306=reports-db.acme.com:3306"
Keep these rules in mind:
- Either side of a mapping can leave out the port.
- AIO tries
host:portfirst, then the host alone. - Hosts you map from are lowercased. Entries with a blank side are dropped.
- A
data_source_hostsvalue passed toaio.initoverrides the environment variable, even when it's empty ({}).
The SDK adds the mapped host to the span as aio.data.source.address, and the port as aio.data.source.port when the mapped value has one. The driver's own address attributes stay as they are, and AIO uses the mapped host in their place.
Understand what's redacted when content capture is off
Content capture is on by default, because aio.init passes capture_message_content=True. It's off when you set capture_message_content to off or set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false.
With content capture off, the SDK keeps the SQL statement but redacts its literals before it sends the span:
- String literals become
'?', and numbers become?. - Strings in
$$quotes become'?'. - Double-quoted strings count as literals only for MySQL, MariaDB, BigQuery, Databricks, Spark, and Hive.
- Comments are kept.
- An unterminated string redacts the rest of the statement.
Table and column names survive redaction, so AIO still detects references from redacted statements.
To redact SQL literals while content capture is on, use aio.redaction_policy(redact_sql_literals=True). The default is False. For more information about redaction, see Redaction overview.
If redaction fails, the SDK replaces content attributes, including the SQL, with REDACTED.
Troubleshooting
AIO records no detected reference from a statement in these cases:
- The database system isn't supported. Use a driver from Supported drivers, or declare the data with
declare_assets. - The span has no host. Make sure the instrumentation records the server address, or map the host with
data_source_hosts. BigQuery spans don't need a host. - The statement can't be parsed. For example, a Postgres statement that uses the
?JSON operator fails to parse. Declare the tables it uses withdeclare_assets.

Have a suggestion?