Worker Tags

Worker tags allow fine-grained routing of shuffle jobs to specific worker subsets. Common use cases include:

  • Resource Isolation: Dedicate workers to specific tenants or applications

  • Hardware Segmentation: Route jobs to SSD-equipped nodes vs. HDD nodes

  • Environment Separation: Separate production and staging worker pools

  • Rolling Upgrades: Tag workers by version for controlled, safe upgrades

Enable Worker Tags

celeborn.tags.enabled=true

Tag Expression Syntax

# AND condition: all tags must match env=production,region=us-east,high-io # Advanced TagsQL mode celeborn.tags.useTagsQL=true env:production region:{us-east,us-west} env:!sandbox

dynamicConfig.yaml — Tag Example

- level: SYSTEM config: celeborn.tags.preferClientTagsExpr: false celeborn.tags.tagsExpr: 'env=production' tags: env=production: - 'worker1:9094' - 'worker2:9094' env=staging: - 'worker3:9094' high-io: - 'worker1:9094' - tenantId: analytics level: TENANT config: celeborn.tags.tagsExpr: 'env=production,high-io'


  Last updated on Apr 23, 2026
Next to read:
Monitoring and Metrics
null
On This Page
Worker TagsEnable Worker TagsTag Expression SyntaxdynamicConfig.yaml — Tag Example