Worker Tags

Worker tags allow fine-grained routing of shuffle jobs to specific worker subsets. Common use cases include:

• Resource Isolation: Dedicate workers to specific tenants or applications
• Hardware Segmentation: Route jobs to SSD-equipped nodes vs. HDD nodes
• Environment Separation: Separate production and staging worker pools
• Rolling Upgrades: Tag workers by version for controlled, safe upgrades

---

Enable Worker Tags

celeborn.tags.enabled=true

---

Tag Expression Syntax

# AND condition: all tags must match
env=production,region=us-east,high-io

# Advanced TagsQL mode

celeborn.tags.useTagsQL=true
env:production region:{us-east,us-west} env:!sandbox

---

dynamicConfig.yaml — Tag Example

- level: SYSTEM

 config:

 celeborn.tags.preferClientTagsExpr: false
 celeborn.tags.tagsExpr: 'env=production'
 tags:

 env=production:

 - 'worker1:9094'
 - 'worker2:9094'
 env=staging:

 - 'worker3:9094'
 high-io:

 - 'worker1:9094'

- tenantId: analytics

 level: TENANT

 config:

 celeborn.tags.tagsExpr: 'env=production,high-io'