ADOC Integrations
Following the Initial ADOC Setup, you can add and configure data sources.
The following list displays the data sources (integrations) supported by ADOC:
- Apache | HDFS
- Apache | Kafka
- Apache | Hive
- Amazon | Athena
- Amazon | S3
- Amazon | Redshift
- Amazon | RDS MariaDB
- Amazon | RDS PostgreSQL
- Amazon | RDS MySQL
- Amazon | Aurora PostgreSQL
- Amazon | Aurora MySQL
- ClickHouse
- (Link Removed)
- IBM | DB2
- Google | BigQuery
- MariaDB
- Microsoft | Azure Blob Storage
- Microsoft | Azure CosmosDB PostgreSQL
- Microsoft | Azure Data Lake Gen2
- (Link Removed)
- Microsoft | Azure MSSQL
- Microsoft | Azure Synapse Analytics
- Microsoft | Power BI
- MongoDB
- Oracle
- Oracle | MySQL
- PostgreSQL
- Snowflake
- SAP | HANA
- Salesforce | Tableau
- SingleStore | MemSQL
- Teradata
- Virtual Data Source
After you have configured your Data Source, ADOC will continually monitor it and give you information, such as how you are utilizing it and other relevant data.
It is possible to produce several instances of the same Data Source using the system's provisioning capabilities.
If you have more than one Snowflake or Azure account, for example, you can follow the instructions in the accompanying image to set up numerous instances of Snowflake or Azure Data Source, one for each of your individual accounts.
To add a Data Source, perform the following steps:
- Click the Add Data Source button in the Register icon tab.
- Follow the instructions provided by the wizard to finalize the setup process on the Select a Data Source page.
For detailed integration steps, explore the relevant sections in the Contents on the left.
The AVRO file format is now supported by ADOC. This feature significantly increases the data format compatibility, allowing users to use AVRO files for greater data observability and dependability.
AVRO Integration and Data Ingestion
Navigate to the Data Source Integration area of ADOC to ingest AVRO file type data. You may simply establish AVRO settings for multiple data storage options, such as ADLS, GCS, S3, and HDFS.
| Field | Description |
|---|---|
| Schema Registry | A registry owned by the user that stores several versions of the AVRO file structure. The exact version of the schema will be used to deserialize the binary AVRO data if available. |
| Schema File | An AVRO schema file is in .avsc format and provides a JSON schema representation. To utilize this, the user must input the path to the .avsc file that will be used for deserialization. |
| Manual Schema | When a user submits a JSON string describing the AVRO schema to be used for de-serialization, it is referred as the manual schema. When configuring the file asset, the user should just copy and paste the JSON string. |
| Embedded | The schema for the data in the AVRO file is included in the file as a JSON string in the header section. This option should be used when the user wants SPARK to infer the schema from the file header. |