Snowflake Reliability

Connecting Snowflake to ADOC helps you monitor your data infrastructure for performance and reliability. Once connected, ADOC can:

• Monitor the health of your Snowflake schemas and datasets
• Track data quality and freshness across tables
• Generate alerts and diagnostics when reliability issues arise

Prerequisites

Before connecting Snowflake to ADOC, you need to set up Snowflake so ADOC can securely monitor your data for quality, performance, and cost. This setup ensures the right users and roles exist, and that they have the permissions needed to access only what’s necessary.

Note All steps in this section must be executed from the Snowflake Web UI or a SQL editor (like SnowSQL or a Snowflake IDE) using an account with ACCOUNTADMIN privileges.

Users and Roles Overview

Before we start, here’s a quick overview of the users and roles that will be created:

ADOC needs access to your actual data — not to change it, but to analyze it for things like freshness, schema drift, and data quality. This part sets up a read-only role and user that ADOC uses to scan your data safely.

Who Should Perform These Steps?

• Snowflake Admins: Set up users, roles, and grants
• Cloud/Network Admins: Configure PrivateLink and DNS
• Data Owners: Choose databases and tables to monitor
• ADOC Users: Complete setup in the ADOC UI

CREATE ROLE IF NOT EXISTS AD_DATA_RELIABILITY_ROLE COMMENT = 'Used by ADOC to monitor data health and quality.';

GRANT USAGE ON WAREHOUSE <your_warehouse_name> TO ROLE AD_DATA_RELIABILITY_ROLE;

GRANT USAGE ON DATABASE <your_database> TO ROLE AD_DATA_RELIABILITY_ROLE;

GRANT USAGE ON SCHEMA <your_database>.<your_schema> TO ROLE AD_DATA_RELIABILITY_ROLE;

GRANT SELECT ON ALL TABLES IN SCHEMA <your_database>.<your_schema> TO ROLE AD_DATA_RELIABILITY_ROLE;

CREATE USER IF NOT EXISTS adoc_user DEFAULT_ROLE = AD_DATA_RELIABILITY_ROLE;

ALTER USER adoc_user SET PASSWORD = '<your_secure_password>';

GRANT ROLE AD_DATA_RELIABILITY_ROLE TO USER adoc_user;

1. Create a Custom Role

This role limits access to just what's needed for monitoring. Using a separate role improves security and auditing.

2. Allow Warehouse Usage

ADOC runs lightweight queries to check data freshness, row counts, and run profiling jobs. It needs a Snowflake warehouse to do this.

3. Grant Access to Your Data

ADOC does not write to your tables — it only needs SELECT access to:

• Profile column statistics
• Validate data quality rules
• Detect schema drift
• Repeat these grants for each database and schema you want ADOC to observe.

Note ADOC uses read-only access and only queries metadata or data you explicitly grant. It never modifies or deletes data and adheres to Snowflake’s least-privilege access model.

4. Create and Configure a Monitoring User

This user is used by ADOC to authenticate and run observability jobs. The user only inherits the read-only permissions from the role you just created.

Add Snowflake as a Data Source

Step 1: Start Setup

1. In the ADOC UI, click Register from the left menu.

2. Click Add Data Source.

3. Select Snowflake.

4. On the Basic Details page:

   1. Enter a name for this data source.
   2. (Optional) Add a description.
   3. Choose your Data Plane or click Setup Data Plane to create one.
   4. You must enable at least one or both of the following to continue:
      • Compute Observability
      • Data Reliability Monitoring

5. Click Next.

Step 2: Add Connection Details

1. Enter the Snowflake URL (e.g., https://<account>.snowflakecomputing.com)

2. Provide the following Snowflake credentials:

   1. Username
   2. Password
   3. Role (e.g., AD_COMPUTE_MONITOR)

3. Select your Data Plane Engine:

   • Spark (for external compute)
   • Pushdown (for in-Snowflake processing)

Note Pushdown is more cost-effective, while Spark provides more control over compute.

4. (Optional) Enable OAuth: If using OAuth, toggle Enable OAuth and provide:

   1. Authorization Endpoint
   2. Token Endpoint
   3. Client ID / Client Secret
   4. (Optional) Enable PKCE

5. (Optional for Pushdown): Configure Global Storage. If Pushdown is selected, you can optionally toggle Configure Global Storage in Snowflake and enter:

   1. Stage Name
   2. Stage File Format (e.g., PARQUET, CSV)

6. Click Test Connection. If the connection is successful, you'll see a “Connected” message. If not, check the credentials and try again.

7. Click Next.

Optimizing Data Partitioning

To tune performance for large datasets, you can adjust Snowflake's default parallelism using this environment variable in your ADOC Data Plane configuration:

SNOWFLAKE_PARTITION_SIZE_IN_MB=10

Note Smaller partitions = higher concurrency = faster data processing

Setting Up PrivateLink (Optional)

You can connect Snowflake to ADOC securely over AWS PrivateLink for improved network isolation.

Prerequisites:

• AWS account with necessary permissions
• VPC in us-west-2
• Snowflake account ready

Step 1: Authorize PrivateLink Access

Why Use PrivateLink?

AWS PrivateLink allows your Snowflake data to connect with ADOC services over a secure, private network path—without traversing the public internet. This enhances data security, reduces latency, and improves performance.

Share your AWS account ID with the Acceldata support team. Acceldata will use this ID to authorize your account for PrivateLink connectivity.

Note This is a one-time setup per AWS account.

Step 2: Create VPC Endpoints

In the AWS Management Console:

1. Navigate to VPC.
2. In the navigation pane, choose Endpoints.
3. Select Create Endpoint.
4. Create the following two endpoints:

Note Make sure the VPC region is set to us-west-2.

Step 3: Configure DNS Using Route 53

In Amazon Route 53:

1. Navigate to the Hosted Zones section for your domain.
2. Add the following A records:

Note These DNS records ensure your traffic is routed directly to the ADOC services via PrivateLink. You’ll need to replace the placeholder values with your actual VPC endpoint IPs.

Security Tip Use least-privileged IAM roles when creating and attaching these endpoints.

Troubleshooting

What’s Next

After you’ve connected Snowflake:

• Visit the Reliability tab to see data quality scores and crawler stats
• Schedule profiling jobs and schema checks
• Get real-time alerts for query spikes, delays, or data freshness issues
• Explore Pipelines to analyze data movement and transformation patterns