Bulk Workflow Migration

To convert all workflows in a single operation, utilize the setup_o2a_bulk.sh script. Ensure that all workflows are placed in a single directory, as demonstrated below.

Bash
    
 
<PARENT-DIR>/             |- workflow1                   |- workflow.xml.   - - Oozie workflow xml                    |- configuration.properties                   |- job.properties             |- workflow2                                 |- workflow.xml    - Oozie workflow xml                    |- configuration.properties                   |- job.properties                   |- ...             - additional folders required to be copied to HDFS
Copy

The setup_o2a_bulk.sh script automates the setup of workflows and their associated configuration files into the required input directory structure, as illustrated below.

Bash
    
[root@airflowdemonode01 o2a]# bash setup_o2a_bulk.sh -f ~/bulk_job[2025-04-25 14:22:39] [INFO] Starting conversion process for /root/bulk_job[2025-04-25 14:22:39] [INFO] Created directory structure in /root/bulk_job_CONVERTED[2025-04-25 14:22:39] [WARN] configuration.properties not found[2025-04-25 14:22:39] [INFO] Processed subworkflow action: hive2_action[2025-04-25 14:22:39] [WARN] configuration.properties not found[2025-04-25 14:22:39] [INFO] Processed shell action: sehaj_errors_sample[2025-04-25 14:22:39] [INFO] Setup completed for /root/bulk_job. You can now run ./run_o2a_bulk.sh to perform the conversion.[2025-04-25 14:22:39] [INFO] Log files are stored in: /root/bulk_job_CONVERTED/logs
Copy

Bash
    
 
<APPLICATION>/             |- job.properties        - job properties that are used to run the job             |- hdfs                  - folder with application - should be copied to HDFS             |     |- workflow.xml    - Oozie workflow xml (1.0 schema)             |     |- ...             - additional folders required to be copied to HDFS             |- configuration.template.properties - template of configuration values used during conversion             |- configuration.properties          - generated properties for configuration values
Copy

Once the directory structure is established, initiate the conversion process by executing the run_o2a_bulk.sh script.

Bash
    
[root@airflowdemonode01 o2a]# ./run_o2a_bulk.sh[2025-04-25 14:22:45] [INFO] Starting bulk conversion process[2025-04-25 14:22:45] [INFO] Converting shell action: sehaj_errors_sample[2025-04-25 14:22:45] [INFO] Input directory: /root/bulk_job_CONVERTED/shell_actions/sehaj_errors_sample[2025-04-25 14:22:45] [INFO] Output directory: /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample[2025-04-25T14:23:40.819+0530] {oozie_converter.py:173} INFO - Adding error handlers[2025-04-25T14:23:40.819+0530] {oozie_converter.py:155} INFO - Converting relations between tasks groups.[2025-04-25T14:23:40.819+0530] {oozie_converter.py:150} INFO - Converting dependencies.[2025-04-25T14:23:40.819+0530] {renderers.py:104} INFO - Saving to file: /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample/sehaj_errors_sample.pyFixing /root/bulk_job_CONVERTED/output/shell_sehaj_errors_sample/sehaj_errors_sample.py[2025-04-25 14:23:41] [INFO] Conversion successful for sehaj_errors_sample[2025-04-25 14:23:44] [INFO] Bulk conversion completed
Copy

After the conversion completes successfully, your Airflow DAG will be available in the specified output directory, such as bulk_job_CONVERTED.

Last updated on May 16, 2025

Was this page helpful?