End to End Flow

Before You Begin

Make sure you contact an Acceldata Support Representative to download the ADOC CLI package. Additionally, you must meet all prerequisites#prerequisites for the usage of ADOC CLI.

Steps to Create UDT Project and Onboard to Torch Platform:

Step 1: Create a Project Using ADOC-CLI Interface

To create a project using ADOC-CLI interface, you first need to install it on your system. Once installed, open a command prompt or terminal and navigate to the directory where you want to create your project. Then, use the new command to create a new project.

Refer the new#new command.

Step 2: Create New UDT Definition

Please refer to the (Link Removed) for information on how to create new UDT definitions and their corresponding test cases.

Step 3: Commit Code Changes to Respective Git Repository

After creating the project, you can commit it to a Git repository to keep track of changes and collaborate with other developers. Create a new Git repository or use an existing one.

Step 4: Create a UDT Driven Manifest File

You must create a manifest file with the information below:

• Torch API and it's credentials
• Storage details: You must provide the storage type, the path where the binaries should be uploaded, and the credentials for the appropriate data storage.
• Package details: This information is regarding the necessary binaries that need to be uploaded and their corresponding meta data.
• UDT definitions: A list of all the UDT definitions and their associated meta data for the Torch platform.

Refer Manifest File Details.

Step 5: Create a CI/CD Pipeline

You must set up a CI/CD pipeline for projects that use UDT. The steps to publish must be configured by the user as a process, followed by:

1. Run test cases: To determine whether the changes are stable, run JUnit test cases using the ADOC CLI test command. Refer the test command.
2. Generate Binaries: To generate binaries, use the ADOC CLI package or build commands. We provide two methods for generating binaries:

• ADOC CLI package command: This command will generate skinny binaries. Refer the package command.
• ADOC CLI build command: This command will generate fat binaries with all the dependencies. Refer the build command.

3. Upload Binaries in accordance with manifest: Using the input manifest file, the ADOC CLI push command will upload the binaries and create all UDT definitions in accordance with the manifest. Refer the push command.

UI Flow:

1. Check UDT listing page for created or updated UDTs

2. Check UDT package list for packages

3. Validate created or updated UDT for a given package in UI

4. Finally, by using the UDTs produced by the repo facility, a Data Quality policy can be created on any asset.

The flow for UDT-driven DQ cycle remains unchanged with regards to output and usage. For more information on how to create a Data Quality policy, see Data Quality Policy.