App NiFi

NiFi Installation

The NiFi Installation capability in xDP allows you to deploy, configure, and manage Apache NiFi directly on your compute clusters. It automates the provisioning of NiFi, a powerful tool for real-time data ingestion, routing, and transformation, abstracting away the complexities of Kubernetes and Helm chart management. This enables your data teams to quickly stand up dataflow automation pipelines, ensuring reliable and governed data movement across your enterprise.

Key Concepts

• Application Catalog: The centralized marketplace within xDP where you can discover, install, and manage the lifecycle of data services like NiFi, Spark, and Trino.
• Helm Chart Configuration: xDP uses Helm, the Kubernetes package manager, to deploy NiFi. The installation wizard provides a user-friendly form and a YAML editor to customize the deployment parameters (known as Helm values), giving you control over resources, networking, and other settings without needing deep Kubernetes expertise.
• Dataflow Automation: This refers to the process of creating, scheduling, and monitoring automated workflows that move and transform data between systems. NiFi is a cornerstone technology for dataflow automation, providing a visual interface to design and manage these data pipelines, which is a critical function for effective Data Governance and Data Operations.

Capabilities

• Simplified Deployment: Provision a production-ready Apache NiFi instance on a selected compute cluster with a few clicks.
• Centralized Configuration: Manage all NiFi deployment settings through the xDP UI, using either a guided form editor for common changes or a full YAML editor for advanced customization.
• Lifecycle Management: Install, monitor, edit, and uninstall NiFi instances from a single, unified interface, streamlining platform administration.

Tutorial (Getting Started)

Prerequisites

• You have an active and healthy Compute Cluster provisioned in xDP.
• Your user role has the necessary permissions to install applications on the target cluster.
• Supported version: NiFi 1.28

Your First Workflow

This tutorial guides you through installing Apache NiFi with its default configuration.

1. From the left navigation menu, go to Platform and select Apps.
2. On the Apps catalog page, locate the Apache NiFi card. Click + Install.
3. On the Release Version screen, review the pre-selected Cluster Release Version and Helm Chart Version. These are typically detected automatically. Click Continue.
4. On the Configure NiFi screen, review the default deployment configuration. For this initial installation, you can accept the defaults.
5. Click Save & Continue at the bottom of the page to start the installation process.
6. The Complete screen confirms that your configuration has been applied and the NiFi installation is in progress on the cluster.

Best Practices

• Version Control for Configurations: For complex or repeatable deployments, use the YAML Editor. Save a copy of your final YAML configuration in a version control system like Git. This practice supports disaster recovery, auditing, and consistent deployments across multiple environments.
• Secure by Default: In the nifiCluster configuration, review the listenersConfig to enable SSL/TLS for the NiFi UI and data ports, ensuring all communication is encrypted.
• Naming Conventions: Use clear and consistent names for your application instances, especially if you plan to run multiple NiFi clusters for different teams or purposes.