Introduction

Acceldata xDP (Cross Data Lake Platform) is a unified data platform designed to run compute workloads across hybrid environments, whether on-premises or in the cloud. It empowers data engineers and data scientists to execute distributed data processing, analytics, and machine learning tasks using popular frameworks like Apache Spark and Jupyter.

Key Features

• Hybrid deployment: Run workloads seamlessly across on-premises data centers and public clouds.
• Managed Control Plane: Centralize orchestration, monitoring, and governance with a fully managed service hosted by Acceldata.
• Customer-hosted Data Plane: Maintain strict data sovereignty by deploying compute clusters directly within your own Kubernetes environment.
• Integrated applications: Develop and execute workloads using built-in support for Apache Spark and interactive Jupyter notebooks.
• Agnostic data integration: Connect natively to a wide range of data stores, including Hadoop (with ODP or others) and cloud object storage such as S3.

Architecture Overview

xDP operates on a secure, split-plane architecture. A centralized Control Plane hosted by Acceldata manages user authentication, orchestration, and metadata. Meanwhile, isolated Data Planes (Compute Clusters) execute the actual data processing directly within your secure network perimeter, ensuring your data never leaves your environment.

User Workflows

xDP supports two primary interaction models for executing workloads:

1. Spark Applications

• API: Programmatically submit jobs and integrate with external orchestration tools.
• UI: Manage, monitor, and execute Spark jobs manually through the xDP web interface.
• Local Connect: Connect your local development environments directly to remote compute clusters for seamless testing.

2. Interactive Notebooks

• Interactive sessions: Perform real-time data exploration, ad-hoc analysis, and machine learning experimentation.
• Scheduled jobs: Automate and schedule your notebook-based workflows for recurring execution.

Frequently Asked Questions (FAQ)

Can I migrate existing Spark applications to xDP? Yes. xDP is fully compatible with standard Spark applications. The platform provides tools and configurations to help you migrate existing workloads with minimal code changes.

What Kubernetes distributions are supported? xDP supports all major Kubernetes distributions, including Amazon EKS, Azure AKS, Google GKE, and various on-premises Kubernetes solutions.

How is data security handled across hybrid environments? xDP enforces consistent security policies across all environments. This includes encryption of data in transit and at rest, granular role-based access controls (RBAC), and comprehensive audit logging.

What are the scaling capabilities of xDP? xDP is architected to handle massive enterprise-level workloads. Your scaling limits are determined only by the capacity of your underlying Kubernetes cluster.