Installing Custom Packages in Apache Airflow
- Access the Node: Log into the node where the Apache Airflow Scheduler and Webserver are installed.
- Navigate to the Airflow Home Directory: Move to the directory designated as the Airflow home.
- Activate the Virtual Environment: Activate the virtual environment used by Airflow to ensure package installations are isolated and consistent.
source <virtual_environment_path>/bin/activate
- Install the Custom Packages: Use the appropriate package manager (e.g.,
pip) to install any required custom packages. For example:
pip install <package_name>- Verify Installation: Ensure the installed packages are available by checking their version or importing them in a Python script.
- Deactivate the Virtual Environment: After installing the packages, deactivate the virtual environment by running:
deactivate- Restart Airflow Services: Restart the Airflow services using the Ambari UI to apply the changes.
Was this page helpful?