Using Ozone S3 Gateway
Ozone offers an S3-compatible REST interface via the Ozone S3 Gateway, enabling users to access object store data using any S3-compatible tools. While the Ozone S3 Gateway is an optional component in standard Ozone installations, within Acceldata's ODP mpack, it is automatically installed and launched alongside the Ozone service. S3 buckets are stored within the /s3v volume.
Prerequisites
To use an S3 endpoint, configuring access key and secret for aws compatible tools is required. The following is an example of awscli.
- Generate Access Key and Secret for AWS : If security is not enabled, you can use any
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY. If security is enabled, you can get the key and the secret with theozone s3 getsecretcommand (Kerberos based authentication is required).
# If security is not enabled :Nothing required. Use any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.# Secure Ozone (kerberos enabled)$ kinit -kt /etc/security/keytabs/ozone.s3g.service.keytab <principal>$ ozone --config /etc/hadoop-ozone/conf/ozone.s3g s3 getsecret --om-service-id=omserviceawsAccessKey=<principal>awsSecret=<new-s3user-secret-key>isDeleted=falsetransactionLogIndex=0- Export these credentials on your S3 endpoint. In the following code block, credentials are updates for a new profile:
# If security is not enabled :$ aws configure AWS Access Key ID [None]: <your-aws-access-ID> AWS Secret Access Key [None]: <your-aws-secret-key> Default region name [None]: <region>Default output format [None]:# Secure Ozone, use credentials from previous step.$ aws configure AWS Access Key ID [None]: <awsAccessKey> AWS Secret Access Key [None]: <awsSecret>Default region name [None]: <region>Default output format [None]:Alternatively, you can create a new profile with Ozone related credentials and use Ozone profile to run S3 utility tasks on awscli.
$ vi /root/.aws/credentials[default]aws_access_key_id = <aws_generated_access_key>aws_secret_access_key = <aws_generated_secret>[ozone]aws_access_key_id = <awsAccessKey>aws_secret_access_key = <awsSecret>- Finally, verify your S3 endpoint from the S3 Gateway UI:
Beginning with Ozone 1.4.0, the generated secret will only be displayed once when generated using getsecret. In the event of the secret being lost, users must first revoke the secret before generating a new one using getsecret.
Ozone S3 Gateway to work with AWS CLI
Ozone S3 Gateway supports various bucket and object operations that the Amazon S3 API provides. Amazon Web Services (AWS) command-line interface (CLI) is one such utility tool, used to interact with S3 Gateway and work with various Ozone storage elements.
Examples of using AWS CLI for Ozone S3 Gateway :
- Create new bucket:
$ aws s3api --endpoint http://odp01.ubuntu.ce:9878 create-bucket --bucket=word OR (depending on how credentials were set up)$ aws s3 ls --endpoint http://odp01.ubuntu.ce:9878 s3://word --profile ozone- Upload key to new bucket:
# Create local file/key to upload on awscat /root/awstpo.txt Hi user,aws file!aws s3 cp --endpoint http://odp01.ubuntu.ce:9878 /root/awstpo.txt s3://wordcount/- Confirm key upload:
aws s3 ls --endpoint http://odp01.ubuntu.ce:9878 s3://wordcount/ 2024-04-16 12:20:20 19 awstpo.txt- Verify file content through Ozone:
# Incase bucket is FSO typeozone --config /etc/hadoop-ozone/conf/ozone.om fs -cat ofs://omservice/s3v/awstestbuck/awstpo.txt# output : 24/04/16 12:22:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable24/04/16 12:22:45 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties24/04/16 12:22:45 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).24/04/16 12:22:45 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system startedHi user,aws file!SSL Enabled Ozone S3 Gateway to Work with AWS CLI
For an SSL-enabled Ozone setup, the S3 Gateway utilizes an HTTPS endpoint. When using Python with the AWS CLI, SSL support is contingent on certificates being in the PEM format. Therefore, ensure your CA certificate is converted to PEM format if it's currently in another format, across all necessary client nodes.
# Validate keystore with required CA certkeytool -list -v -keystore <ssl.client.truststore.location># Export CA cert to truststore and PEM formatkeytool -export -alias <alias> -file <s3g-ca.crt> -keystore <ssl.client.truststore.location>openssl x509 -inform DER -outform PEM -in <s3g-ca.crt> -out /path/to/s3gca.pem# Configure aws credentials as per steps in prerequisite.aws configurePass the certificate in PEM file format to the aws s3api commands to perform S3 utility tasks. For example:
- Create a new bucket:
aws --debug s3api --endpoint https://odp01.ubuntu.ce:9879 --ca-bundle "/tpo/attempt2/ca-cert.pem" create-bucket --bucket=word- Upload the key to the new bucket:
# Create local file/key to upload on awscat /root/awstpo.txt Hi user,aws file! aws s3 cp --endpoint https://odp01.ubuntu.ce:9879 /root/awstpo.txt s3://wordcount/ --ca-bundle "/tpo/attempt2/ca-cert.pem"- Confirm the key upload:
aws s3 ls --endpoint https://odp01.ubuntu.ce:9879 s3://wordcount/ --ca-bundle "/tpo/attempt2/ca-cert.pem"2024-04-16 12:20:20 19 awstpo.txt- Verify the file content through Ozone:
cd /usr/odp/3.2.3.3-2/ozone/bin./ozone --config /etc/hadoop-ozone/conf/ozone.om fs -cat ofs://omservice/s3v/awstestbuck/awstpo.txt# output : 24/04/16 12:22:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable24/04/16 12:22:45 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties24/04/16 12:22:45 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).24/04/16 12:22:45 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system startedHi user,aws file!Revoke Access to Generated AWS Credentials
Revoke access to AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY once your use case is completed.
ozone --config /etc/hadoop-ozone/conf/ozone.s3g s3 revokesecret24/03/08 18:19:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable24/03/08 18:19:31 INFO client.ClientTrustManager: Loading certificates for client.Enter 'y' to confirm S3 secret revocation for 's3g/odp01.ha.ubuntu.ce@ADSRE.COM': yS3 secret revoked.