AVRO Integrations
The AVRO file format is now supported by ADOC. This feature significantly increases the data format compatibility, allowing users to use AVRO files for greater data observability and dependability.
AVRO Integration and Data Ingestion
Navigate to the Data Source Integration area of ADOC to ingest AVRO file type data. You may simply establish AVRO settings for multiple data storage options, such as ADLS, GCS, S3, and HDFS.
| Field | Description |
|---|---|
| Schema Registry | A registry owned by the user that stores several versions of the AVRO file structure. The exact version of the schema will be used to deserialize the binary AVRO data if available. |
| Schema File | An AVRO schema file is in .avsc format and provides a JSON schema representation. To utilize this, the user must input the path to the .avsc file that will be used for deserialization. |
| Manual Schema | When a user submits a JSON string describing the AVRO schema to be used for de-serialization, it is referred as the manual schema. When configuring the file asset, the user should just copy and paste the JSON string. |
| Embedded | The schema for the data in the AVRO file is included in the file as a JSON string in the header section. This option should be used when the user wants SPARK to infer the schema from the file header. |
Was this page helpful?