Acceldata
ODP

Last updated: Sep 24, 2026 16:08 UTC

Worker Tags

Worker tags allow fine-grained routing of shuffle jobs to specific worker subsets. Common use cases include:

  • Resource Isolation: Dedicate workers to specific tenants or applications
  • Hardware Segmentation: Route jobs to SSD-equipped nodes vs. HDD nodes
  • Environment Separation: Separate production and staging worker pools
  • Rolling Upgrades: Tag workers by version for controlled, safe upgrades


Enable Worker Tags

celeborn.tags.enabled=true


Tag Expression Syntax

# AND condition: all tags must match
env=production,region=us-east,high-io

# Advanced TagsQL mode

celeborn.tags.useTagsQL=true
env:production region:{us-east,us-west} env:!sandbox


dynamicConfig.yaml — Tag Example

- level: SYSTEM

  config:

    celeborn.tags.preferClientTagsExpr: false
    celeborn.tags.tagsExpr: 'env=production'
  tags:

    env=production:

      - 'worker1:9094'
      - 'worker2:9094'
    env=staging:

      - 'worker3:9094'
    high-io:

      - 'worker1:9094'

- tenantId: analytics

  level: TENANT

  config:

    celeborn.tags.tagsExpr: 'env=production,high-io'