Amazon | Redshift

Redshift is AWS' data warehouse service. You can use Redshift to analyze structured and semi-structured data. Redshift uses SQL for data analysis. You can analyze warehouses, operational databases, and data lakes using Redshift. Redshift uses ML (machine learning) for data analysis.

Redshift in ADOC

ADOC provides data reliability capability for data stored in your Redshift data source. You must create a Data Plane or use an existing Data Plane to add Redshift as a Data source in ADOC. Once you add Redshift as a Data Source, you can view the details of your Redshift usage in the Data Reliability tab in ADOC.

Steps to Add Redshift as a Data Source

To add Redshift as a Data source:

1. Click Register from the left pane.
2. Click Add Data Source.
3. Select the Redshift Data Source. Redshift Data source basic Details page is displayed.

4. Enter a name for the data source in the Data Source name field.
5. (Optional) Enter a description for the Data Source in the Description field.
6. Enable the Data Reliability capability by switching on the toggle switch.
7. Select a Data Plane from the Select Data Plane drop-down menu.

To create a new Data Plane, click Setup Dataplane.

8. Click Next. The Redshift Connection Details page is displayed.

9. Enter the JDBC URL of your Redshift account in the Redshift JDBC URL field. To learn more about Redshift JDBC URL, refer to this AWS document.
10. Enter the JDBC username of your Redshift account in the Redshift Username field.
11. Enter the JDBC password of your Redshift account in the Redshift Password field.
12. Click Test Connection. If your credentials are valid, you receive a Connected message. If you get an error message, validate the Redshift credentials you entered.
13. Click Next. The Observability Setup page is displayed.
14. Choose the required options on the Observability setup page.
15. Enable Crawler Execution Schedule: Turn on this toggle switch to select a time tag and time zone to schedule the execution of crawlers for Data Reliability.
16. Click Submit.

Once you successfully add Databricks as a data source, you can navigate to the data source page to view the options available after adding the Data Source.

Configure IAM Authentication for Redshift

To securely connect ADOC to your Amazon Redshift cluster, you must configure IAM authentication with the appropriate permissions.

1. Create an IAM Role:

Create an IAM role to manage temporary credentials for the data-plane services running on an EC2 instance that will access the Redshift resources.

Note This is preferable to storing access keys on an EC2 instance.

2. Create an Inline Policy with Minimum Permissions

2.1. Define the policy with minimum permissions for the IAM role to use the GetClusterCredentials function. This activity is required for the data plane to authenticate and manage access on behalf of users.

arn:aws:redshift:region:account-id:dbuser:cluster-name/dbuser-name

​

eg: arn:aws:redshift:ap-south-1:079776845623:dbuser:redshift-cluster-706-2/awsuser

Specifying the Amazon Redshift dbuser resource allows the role access to the Redshift cluster's database user, temp_creds_user. The data-plane requires authority to perform the GetClusterCredentials function on behalf of the users.

2.2. Add an ARN that specifies the Amazon Redshift dbname resource in the following format.

arn:aws:redshift:region:account-id:dbname:cluster-name/database-name

​

eg: arn:aws:redshift:ap-south-1:079776845623:dbname:redshift-cluster-706-2/dev

Replace region, account-id, and cluster-name with the respective values for your AWS Region, Account, and Cluster. For database-name, enter the name of the database where the user will log in.

// BASE MINIMUM PERMISSIONS FOR AWS REDSHIFT WITH IAM

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Sid": "RedshiftBaseMinimumAccess",

            "Effect": "Allow",

            "Action": [

                "redshift:GetClusterCredentials"

            ],

            "Resource": [

                "arn:aws:redshift:<region>:<account-id>:dbuser:<cluster-name>/<dbuser-name>",

                "arn:aws:redshift:<region>:<account-id>:dbname:<cluster-name>/<database-name>"

            ]

        }

    ]

}

3. Attach the IAM role to Data Plane Services

There are two options to attach the roles created:

Option 1: EC2 Instance Profile Authentication:

1. Attach the policy to the role assigned to the EC2 instance where the data-plane service is running.
2. Create an instance profile that includes the IAM role and assign it to the EC2 instance.

Option 2: AWS IAM Role for Service Accounts:

1. Attach the policy to the role and annotate the necessary data-plane service accounts with the role ARN.
2. Use the specific ARNs mentioned above to grant the necessary permissions.