Rules and Rule Sets

In ADOC, Rules and Rule Sets help ensure your datasets meet business data quality standards. Rules define specific checks on data, and Rule Sets group multiple rules for easy application, scheduling, and monitoring.

What Are Rules?

A Rule is a specific condition or check that data must meet to be considered accurate, complete, and reliable. Each rule focuses on a particular aspect of data quality, such as null values, uniqueness, value ranges, patterns, or schema compliance.

Key Points:

• Rules are tag-based. They only apply to assets with matching tags.
• A rule cannot be applied directly to a data source; it must be part of a Rule Set.
• Rules can be created, edited, duplicated, or deleted.

Example: A Customer Email Validation rule might include:

• Rule Type: Null Check
• Description: Ensure customer_email is not empty
• Tag: CustomerData

What Are Rule Sets?

A Rule Set is a collection of rules that are applied together to a dataset or asset.

• Rule Sets allow multiple rules to run as a batch.
• Applying a Rule Set automatically generates policies for monitoring.
• Rule Sets can be scheduled for repeated execution.
• Rules inside a Rule Set are executed together; policies track which rules pass or fail.

Example: Rule Set: Customer Data Quality Checks

• Rules Included:

  • Customer Email Validation (Null Check)
  • Customer ID Uniqueness (Duplicate Row Check)
  • Customer Phone Validation (Pattern Match)

• The above rules are then applied to all assets tagged CustomerData.

Creating a Rule

Step 1: Start Rule Creation

1. Navigate to Governance -> Policies -> Rules / RuleSets from the left menu.
2. Click Create Rule.

Note Users with view-only permissions cannot create rules or rule sets.

Step 2: Configure Rule Details

Tag Match Example:

• All: Only assets matching all tags are evaluated.
• Any: Assets matching at least one tag are evaluated.

Step 3: Save the Rule

Click Save.

The rule now appears in the Rules Table.

Rules Table Overview

Creating a Rule Set

Step 1: Start Rule Set Creation

1. Navigate to Data Reliability -> Manage Policies -> Rule/Rule Sets -> Rule Sets tab.
2. Click Create Rule Set.

Step 2: Configure Rule Set Details

Step 3: Save the Rule Set

Click Save.

The Rule Set now appears in the Rule Sets Table.

Rule Sets Table Overview

Importing a Rule Set

1. Click Import Rule Set.

2. Drag-and-drop a .zip or .gz file or browse your system.

3. Handle duplicates:

   • Override: Replace existing Rule Set
   • Do Not Import: Skip duplicates

4. Click Next and wait for import to complete.

Note If Do Not Import is selected, no duplicates are imported.

Applying a Rule Set

Applying a rule set creates Data Quality policies in bulk across a selected set of assets. ADOC evaluates the tag matching criteria for each rule in the rule set and creates the corresponding policies on all qualifying assets.

When applying a rule set, you can configure a policy execution schedule and notification channels as part of the apply flow. All policies created by the run will inherit these settings.

1. On the Rule Sets page, locate the rule set you want to apply, and click the ellipsis icon in the last column.
2. From the drop-down menu, select Apply Rule Set.
3. On the Apply Rule Set page, select the Asset Type you want to apply the rule set to.
4. Select the assets on which the policies should be created. ADOC displays an estimated count of assets that match your selection.
5. Under Policy Schedule, select a schedule to define when the created policies should execute. This field is optional. If no schedule is selected, the policies are created without an execution schedule.
6. Under Notifications, select one or more notification channels to attach to the policies being created. This field is optional. If no channels are selected, the policies are created without notification configuration.
7. Review the summary of your selections and click Apply Ruleset.

ADOC runs an asynchronous job to process the rule set application. You can monitor the progress and results under Rule Set Jobs.

Example:

• Applying Customer Data Quality Checks to a database automatically creates policies for all included rules.
• Users without sufficient permissions may not see the created policies.

Scheduling a Rule Set

When scheduling a rule set, you can configure a policy execution schedule and notification channels as part of the apply flow. All policies created by the run will inherit these settings.

1. On the Rule Sets page, locate the rule set you want to schedule, and click the ellipsis icon in the last column.
2. From the drop-down menu, select Schedule Rule Set.
3. On the Schedule Rule Set page, enter a name for the schedule and select the Asset Type you want to apply the rule set to. Select the assets on which the policies should be created, and click Next.
4. Under Rule Set Schedule, select a schedule to configure when the rule set will execute.
5. Under Policy Schedule, select a schedule to define when the created policies should execute. This field is optional. If no schedule is selected, the policies are created without an execution schedule.
6. Under Notifications, select one or more notification channels to attach to the policies being created. This field is optional. If no channels are selected, the policies are created without notification configuration.
7. Review the summary of your selections and click Create Schedule.

Viewing Rules and Schedules

• View Rules: Click the ellipsis > View Rules. Shows rule name, type, applied by, and creation date.
• View Schedules: Click ellipsis > View Schedules. Shows schedule name, frequency, and enabled status.

Example

Scenario: Monitor customer data quality.

Step 1: Create Rules

• Customer Email Validation (Null Check)
• Customer ID Uniqueness (Duplicate Row Check)
• Customer Phone Validation (Pattern Match)

Step 2: Create a Rule Set

• Customer Data Quality Checks (includes the 3 rules above)

Step 3: Apply the Rule Set to Assets

• Assets tagged CustomerData
• Result: Applying the Rule Set automatically creates a Data Quality Policy for each asset.

Step 4: Schedule the Policy

• Schedule the automatically created policy to run daily.
• The policy will execute all rules in the Rule Set, track pass/fail results, and generate alerts if necessary.

Outcome:

• The Policy monitors customer data continuously, not the Rule Set itself
• Reports and alerts indicate which rules passed or failed for each asset