Shell Example
Prerequisites
Make sure to copy examples/shell/configuration.template.properties, rename it as configuration.properties and fill with configuration data.
Run the Command
To execute the Shell example using the Oozie-to-Airflow (o2a) converter, run the following command:
o2a -i examples/shell -o output/shellMany shell action jobs invoke Bash scripts to execute Spark or Hive jobs. These Bash scripts are often stored in the Hadoop Distributed File System (HDFS) and executed within YARN containers. This highlights a core difference between Oozie and Airflow: Oozie workflows run in YARN containers within specific job queues, whereas Airflow DAGs execute on the Airflow worker nodes themselves. Consequently, Airflow workers do not have direct access to HDFS, necessitating the copying of required files to the local working directory for execution.
Output
In this example the output is created in the ./output/shell/ folder.
The converted DAG utilizes Airflow's BashOperator, which executes the specified shell action directly on the Airflow worker node.