Gluten with Velox

Apache Gluten with Velox backend is a native execution plugin for Apache Spark that accelerates SQL query performance without requiring any code changes to your existing Spark applications.

Key Benefits

Zero Code Changes – Works with existing Spark SQL, DataFrame, and Dataset APIs
Significant Performance Gains – 2–5× faster on analytical workloads
Drop-in Replacement – Simply add JAR and configuration
Transparent Acceleration – Automatically offloads supported operations to native Velox engine
Production Ready – Battle-tested on TPC-DS, TPC-H benchmarks

What is Gluten?

Apache Gluten is a Spark plugin that offloads SQL query execution from JVM to native C++ execution engines. It acts as a “glue” layer between Apache Spark and vectorized execution engines like Velox.

Architecture Overview

Why Velox?

Velox is Meta's unified execution engine that powers:

Meta's data warehouse (Presto)
Stream processing systems
Machine learning infrastructure

Key Features

Vectorized execution – processes data in batches
SIMD instructions – hardware-accelerated operations
Advanced codegen – runtime code generation
Memory efficiency – Arrow-based zero-copy design

How It Works

Transparent Query Acceleration

When you enable Gluten, Spark automatically:

Analyzes your SQL query plan
Identifies operations that can be accelerated (scans, filters, joins, aggregations)
Offloads those operations to the Velox native engine
Falls back to Spark for unsupported operations
Returns results – your application receives the same output, just faster!

Supported Operations

✅ Fully Supported

Table scans (Parquet, ORC, CSV)
Filters and predicates
Projections
Hash joins (inner, left, right, full outer, semi, anti)
Hash aggregations (sum, avg, min, max, count)
Sort operations
Window functions
String operations
Date/time operations
Math functions

⚠️ Partial Support

Some complex UDFs
Certain window function combinations
Specialized data types

❌ Not Supported

Custom data sources
Complex nested UDFs
Some exotic data types

Performance Expectations

Based on TPC-DS and production workloads:

Workload Type	Expected Speedup
Scan-heavy queries	2–3×
Join-heavy queries	3–5×
Aggregation-heavy	2–4×
Complex analytics	2–3×
Simple queries	1.5–2×

Real-world Examples

TPC-DS Query 72: 45s → 12s (3.75× speedup)
TPC-DS Query 95: 120s → 35s (3.4× speedup)

Getting Started

Prerequisites

Component	Version
Apache Spark	3.5.x
Java	8 or 11 (upcoming release)
Operating System	Linux x86_64
CPU	AVX2 support recommended

Step 1: Obtain Gluten JAR

Contact your Acceldata account manager or download from:

Place JAR in:

Step 2: Enable Gluten (No Code Changes!)

Option A: Via `spark-submit`

Bash
    
spark-submit   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox   your-application.jar
Copy

Option B: Via `spark-defaults.conf`

Add to $SPARK_HOME/conf/spark-defaults.conf:

Option C: Programmatic Configuration (Scala/Java)

Scala
    
​x
 
val spark = SparkSession.builder()  .appName("MyApp")  .config("spark.plugins", "org.apache.gluten.GlutenPlugin")  .config("spark.memory.offHeap.enabled", "true")  .config("spark.memory.offHeap.size", "8g")  .config("spark.gluten.sql.enabled", "true")  .config("spark.gluten.sql.columnar.backend.lib", "velox")  .getOrCreate()​// Your existing code runs unchanged!spark.sql("SELECT * FROM large_table WHERE value > 100").show()
Copy

Option D: PySpark

Python
    
from pyspark.sql import SparkSession​spark = SparkSession.builder     .appName("MyApp")     .config("spark.plugins", "org.apache.gluten.GlutenPlugin")     .config("spark.memory.offHeap.enabled", "true")     .config("spark.memory.offHeap.size", "8g")     .config("spark.gluten.sql.enabled", "true")     .config("spark.gluten.sql.columnar.backend.lib", "velox")     .getOrCreate()​# Your existing code runs unchanged!df = spark.sql("SELECT * FROM large_table WHERE value > 100")df.show()
Copy

Step 3: Run Your Application

That's it! Your application runs exactly as before, but faster.

Configuration Guide

Minimal Configuration (Quick Start)

For JDK 11

Bash
    
spark-submit   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox   --conf spark.driver.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"   --conf spark.executor.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"   your-application.jar
Copy

For JDK 8

Bash
    
spark-submit   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox   your-application.jar
Copy

Note: JDK 8 does not require the --add-opens parameters.

Recommended Production Configuration

Bash
    
spark-submit   --master yarn   --deploy-mode cluster   --driver-memory 4g   --executor-memory 16g   --executor-cores 4   --num-executors 10   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar     # Core Gluten Settings  --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox     # Memory Configuration  --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=12g   --conf spark.gluten.memory.fraction=0.7     # Shuffle Configuration  --conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager   --conf spark.gluten.sql.columnar.shuffle.codec=lz4     # Adaptive Query Execution  --conf spark.sql.adaptive.enabled=true   --conf spark.sql.adaptive.coalescePartitions.enabled=true   --conf spark.sql.adaptive.skewJoin.enabled=true     # Velox-Specific Settings  --conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true   --conf spark.gluten.sql.columnar.backend.velox.spillPath=/tmp/velox-spill     # Java Options (JDK 11 only)  --conf spark.driver.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"   --conf spark.executor.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"     your-application.jar
Copy

TPC-DS Benchmark Configuration (Tested)

This is the exact configuration used by Acceldata for TPC-DS benchmarking.

For JDK 11

Bash
    
spark-sql   --driver-memory 3g   --executor-memory 12g   --executor-cores 3   --num-executors 9   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.sql.optimizer.runtime.bloomFilter.applicationSideScanSizeThreshold=128MB   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox   --conf spark.gluten.sql.columnar.backend.velox=true   --conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager   --conf spark.gluten.sql.columnar.shuffle.codec=lz4   --conf spark.driver.extraJavaOptions="--illegal-access=permit -Dio.netty.tryReflectionSetAccessible=true --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"   --conf spark.executor.extraJavaOptions="--illegal-access=permit -Dio.netty.tryReflectionSetAccessible=true --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED"   --conf spark.sql.adaptive.enabled=true   --conf spark.sql.adaptive.coalescePartitions.enabled=true   --conf spark.sql.adaptive.skewJoin.enabled=true   --conf spark.sql.adaptive.localShuffleReader.enabled=true   --conf spark.gluten.sql.columnar.backend.velox.memCacheSize=4g   --conf spark.sql.inMemoryColumnarStorage.compressed=true   --conf spark.sql.inMemoryColumnarStorage.batchSize=50000   --conf spark.gluten.sql.columnar.forceShuffledHashJoin=false   --conf spark.gluten.memory.fraction=0.7   --conf spark.gluten.sql.columnar.backend.velox.maxBatchSize=32768   --conf spark.gluten.sql.columnar.backend.velox.bloomFilterEnabled=true   --conf spark.gluten.sql.columnar.preferColumnar=true   --conf spark.gluten.sql.columnar.backend.velox.aggregationPreferredSize=1048576   --conf spark.sql.adaptive.advisoryPartitionSizeInBytes=128MB   --conf spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold=1GB   --conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true   --conf spark.gluten.sql.columnar.backend.velox.spillPath=/tmp/velox-spill   --conf spark.serializer=org.apache.spark.serializer.KryoSerializer   --conf spark.sql.execution.arrow.pyspark.enabled=true   --database tpcds_parquet   -f file:///home/hive/hive-testbench/spark-queries-tpcds/query42.sql
Copy

For JDK 8

Bash
    
spark-sql   --driver-memory 3g   --executor-memory 12g   --executor-cores 3   --num-executors 9   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-spark3.5_2.12-linux_amd64-1.4.0.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.sql.optimizer.runtime.bloomFilter.applicationSideScanSizeThreshold=128MB   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --conf spark.gluten.sql.columnar.backend.lib=velox   --conf spark.gluten.sql.columnar.backend.velox=true   --conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager   --conf spark.gluten.sql.columnar.shuffle.codec=lz4   --conf spark.sql.adaptive.enabled=true   --conf spark.sql.adaptive.coalescePartitions.enabled=true   --conf spark.sql.adaptive.skewJoin.enabled=true   --conf spark.sql.adaptive.localShuffleReader.enabled=true   --conf spark.gluten.sql.columnar.backend.velox.memCacheSize=4g   --conf spark.sql.inMemoryColumnarStorage.compressed=true   --conf spark.sql.inMemoryColumnarStorage.batchSize=50000   --conf spark.gluten.sql.columnar.forceShuffledHashJoin=false   --conf spark.gluten.memory.fraction=0.7   --conf spark.gluten.sql.columnar.backend.velox.maxBatchSize=32768   --conf spark.gluten.sql.columnar.backend.velox.bloomFilterEnabled=true   --conf spark.gluten.sql.columnar.preferColumnar=true   --conf spark.gluten.sql.columnar.backend.velox.aggregationPreferredSize=1048576   --conf spark.sql.adaptive.advisoryPartitionSizeInBytes=128MB   --conf spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold=1GB   --conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true   --conf spark.gluten.sql.columnar.backend.velox.spillPath=/tmp/velox-spill   --conf spark.serializer=org.apache.spark.serializer.KryoSerializer   --conf spark.sql.execution.arrow.pyspark.enabled=true   --database tpcds_parquet   -f file:///home/hive/hive-testbench/spark-queries-tpcds/query42.sql
Copy

Performance Tuning – Core Gluten Parameters

1. Memory Configuration

spark.memory.offHeap.enabled – enable off-heap memory
spark.memory.offHeap.size – off-heap memory size (recommend 8–12g per executor)
spark.gluten.memory.fraction – fraction of off-heap for Gluten (recommend ~0.7)

Rule of thumb

Off-heap memory ≈ 60–75% of executor memory Example: executor-memory=16g → offHeap.size=10–12g

2. Shuffle Configuration

spark.shuffle.manager = org.apache.spark.shuffle.sort.ColumnarShuffleManager
spark.gluten.sql.columnar.shuffle.codec = lz4 (or zstd)

3. Velox Backend Settings

Key parameters to tune:

spark.gluten.sql.columnar.backend.velox.maxBatchSize = 16384–32768
spark.gluten.sql.columnar.backend.velox.memCacheSize = 2–4g
spark.gluten.sql.columnar.backend.velox.aggregationPreferredSize = 1048576–4194304
spark.gluten.sql.columnar.backend.velox.bloomFilterEnabled = true (join-heavy workloads)

4. Spilling Configuration

Bash
    
--conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true --conf spark.gluten.sql.columnar.backend.velox.spillPath=/mnt/ssd/velox-spill
Copy

5. Adaptive Query Execution (AQE)

Bash
    
 
--conf spark.sql.adaptive.enabled=true--conf spark.sql.adaptive.coalescePartitions.enabled=true--conf spark.sql.adaptive.skewJoin.enabled=true--conf spark.sql.adaptive.localShuffleReader.enabled=true--conf spark.sql.adaptive.advisoryPartitionSizeInBytes=128MB
Copy

Monitoring and Validation

Verify Gluten is Active

Method 1 – Spark UI

Open Spark UI: http://<driver-host>:4040
Environment tab → check:
spark.plugins = org.apache.gluten.GlutenPlugin
spark.gluten.sql.enabled = true

Method 2 – Query Plan

Scala
    
 
val df = spark.sql("SELECT * FROM large_table WHERE id > 100")df.explain(true)// Look for: GlutenColumnarToRow, GlutenScan, GlutenFilter, GlutenProject
Copy

Python
    
 
df = spark.sql("SELECT * FROM large_table WHERE id > 100")df.explain(True)# Look for Gluten operators in the physical plan
Copy

Method 3 – Logs

Bash
    
 
grep -i "gluten" /var/log/spark/spark-driver.log# "GlutenPlugin is loaded"# "Velox backend is initialized"
Copy

Performance Metrics

Query execution time (Spark UI → SQL tab)
Shuffle read/write size and time
GC time
Off-heap memory usage and spilling
CPU utilization

Example comparison

Bash
    
# Without Glutenspark-sql --database mydb -e "SELECT COUNT(*) FROM large_table"​# With Glutenspark-sql   --jars gluten-velox-bundle-*.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --database mydb -e "SELECT COUNT(*) FROM large_table"
Copy

Benchmark Your Workload

Bash
    
#!/bin/bash# Baseline (without Gluten)echo "=== Running WITHOUT Gluten ==="time spark-sql --database mydb -f my_query.sql > baseline.out​# With Glutenecho "=== Running WITH Gluten ==="time spark-sql   --jars /usr/odp/current/spark3-client/jars/gluten-velox-bundle-*.jar   --conf spark.plugins=org.apache.gluten.GlutenPlugin   --conf spark.memory.offHeap.enabled=true   --conf spark.memory.offHeap.size=8g   --conf spark.gluten.sql.enabled=true   --database mydb -f my_query.sql > gluten.out​# Compare resultsdiff baseline.out gluten.out  # Should be identical
Copy

Best Practices

1. Data Format Recommendations

Best performance

Parquet (columnar, compressed)
ORC (columnar, compressed)

Good

CSV (large files)
JSON (structured)

Poor

Small files (<128MB)
Plain uncompressed text

SQL
    
 
-- Convert to Parquet for best resultsCREATE TABLE optimized_tableUSING PARQUETAS SELECT * FROM original_table;
Copy

2. Partition Strategy

Partition by date/time for time-series
Aim for 128–256MB file/partition size
Avoid thousands of tiny files

SQL
    
 
-- Good partitioningCREATE TABLE events (  id BIGINT,  event_time TIMESTAMP,  data STRING)USING PARQUETPARTITIONED BY (date DATE)LOCATION '/data/events';
Copy

3. Query Optimization

Use WHERE predicates for pushdown
Use partition pruning
SELECT only needed columns
Enable bloom filters for joins

Avoid

SELECT * on wide tables
Complex nested UDFs
Excessive small shuffles

4. Resource Allocation

Bash
    
 
# Small jobs (< 1TB)--executor-memory 8g--executor-cores 4--conf spark.memory.offHeap.size=6g​# Medium jobs (1–10TB)--executor-memory 16g--executor-cores 4--conf spark.memory.offHeap.size=12g​# Large jobs (> 10TB)--executor-memory 32g--executor-cores 8--conf spark.memory.offHeap.size=24g
Copy

5. Fallback Strategy

Bash
    
 
# Disable Gluten if needed--conf spark.gluten.sql.enabled=false# Or omit Gluten JAR and plugin configs entirely
Copy

Troubleshooting

Issue 1: “Plugin GlutenPlugin could not be loaded”

Symptoms

Fix

Verify JAR path:

Bash
    
 
ls -lh /usr/odp/current/spark3-client/jars/gluten-velox-bundle-*.jar
Copy

Ensure JAR is included in --jars
Check Spark version compatibility (Gluten 1.4.0 ↔ Spark 3.5.x)

Issue 2: Native Library Load Failure

Symptoms

Fix

Bash
    
 
unzip -l gluten-velox-bundle-*.jar | grep "\.so$"cat /proc/cpuinfo | grep avx2   # check AVX2 support
Copy

Issue 3: Out of Memory (OOM)

Fix

Increase off-heap:

Bash
    
 
--conf spark.memory.offHeap.size=16g
Copy

Enable spilling:

Bash
    
--conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true --conf spark.gluten.sql.columnar.backend.velox.spillPath=/tmp/velox-spill
Copy

Reduce batch size:

Bash
    
 
--conf spark.gluten.sql.columnar.backend.velox.maxBatchSize=16384
Copy

Issue 4: Result Mismatch vs Vanilla Spark

Fix

Check data types (decimals, timestamps)
Ensure query is supported by Gluten
Temporarily disable Gluten:

Scala
    
 
spark.conf.set("spark.gluten.sql.enabled", "false")// run queryspark.conf.set("spark.gluten.sql.enabled", "true")
Copy

Issue 5: Slower with Gluten

Very small data (<100MB) → use vanilla Spark
Excessive ColumnarToRow conversions → inspect plan and simplify query
Increase spark.memory.offHeap.size
Tune spark.gluten.sql.columnar.shuffle.codec (try lz4 or zstd)

Debugging Tips

Enable more logging:

Bash
    
 
--conf spark.gluten.sql.columnar.backend.velox.logLevel=INFO --conf spark.sql.adaptive.logLevel=DEBUG
Copy

Check plans:

Scala
    
 
val df = spark.sql("YOUR_QUERY")df.explain(mode = "extended")df.queryExecution.debug.codegen()
Copy

Inspect Gluten-related configs:

Scala
    
 
spark.sparkContext.getConf.getAll  .filter(_._1.contains("gluten"))  .foreach(println)
Copy

FAQ

General

Q: Do I need to rewrite my Spark applications? A: No. Gluten works transparently with existing Spark SQL, DataFrame, and Dataset code.

Q: Will query results be the same? A: Yes. If you see differences, treat them as bugs and escalate.

Q: Does Gluten work with PySpark / SparkR / Scala / Java? A: Yes.

Q: Does Gluten support UDFs? A: Built-in functions are supported; custom UDFs usually fall back to Spark.

Q: What about Structured Streaming? A: Works for SQL operations in micro-batch mode.

Performance

Typical speedup: 2–5× on analytical workloads
Limited gains on very small queries (<100MB)
Fully compatible with dynamic allocation

Technical

File formats: Parquet, ORC, CSV, JSON (best with Parquet / ORC)
Works with Hive metastore and Hive tables
Compatible with Delta Lake / Iceberg / Hudi through Spark
Storage: S3, HDFS, ADLS, GCS, etc.

Disable for a specific query:

Scala
    
 
spark.conf.set("spark.gluten.sql.enabled", "false")// run queryspark.conf.set("spark.gluten.sql.enabled", "true")
Copy

Compatibility & Deployment

Spark versions: 3.3.x, 3.4.x, 3.5.x (Gluten 1.4.0)
Java: 8 and 11
Cluster managers: YARN, Kubernetes, Standalone
Arch: x86_64 (ARM64 planned)

Rollback: remove Gluten JAR + plugin configs. Works in notebooks (Jupyter, Zeppelin) via Spark conf.

Quick Reference

Essential Config

Bash
    
 
# Minimum--jars gluten-velox-bundle-*.jar--conf spark.plugins=org.apache.gluten.GlutenPlugin--conf spark.memory.offHeap.enabled=true--conf spark.memory.offHeap.size=8g--conf spark.gluten.sql.enabled=true--conf spark.gluten.sql.columnar.backend.lib=velox​# Recommended--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager--conf spark.sql.adaptive.enabled=true--conf spark.gluten.sql.columnar.backend.velox.spillEnabled=true​# JDK 11 only--conf spark.driver.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED"--conf spark.executor.extraJavaOptions="--add-opens java.base/java.lang=ALL-UNNAMED"
Copy

Quick Checks

Bash
    
 
# Plugin loaded?grep "GlutenPlugin" spark-driver.log​# Plan shows Gluten operators?spark.sql("YOUR_QUERY").explain()# Look for GlutenScan / GlutenFilter / GlutenProject
Copy

Appendix – Key Gluten Parameters

spark.gluten.sql.enabled – master enable/disable switch
spark.gluten.sql.columnar.backend.lib – backend engine (velox)
spark.gluten.memory.fraction – fraction of off-heap for Gluten
spark.gluten.sql.columnar.preferColumnar – prefer columnar exec
spark.gluten.sql.columnar.forceShuffledHashJoin – force hash join
spark.gluten.sql.columnar.shuffle.codec – shuffle codec (lz4)
spark.gluten.sql.columnar.backend.velox.maxBatchSize – rows per batch
spark.gluten.sql.columnar.backend.velox.memCacheSize – cache size
spark.gluten.sql.columnar.backend.velox.spillEnabled – enable spilling
spark.gluten.sql.columnar.backend.velox.spillPath – spill directory
spark.gluten.sql.columnar.backend.velox.bloomFilterEnabled – bloom filters
spark.gluten.sql.columnar.backend.velox.aggregationPreferredSize – aggregation table size

Support and Resources

Apache Gluten GitHub – project source and issues
Velox GitHub – execution engine documentation
Apache Spark Tuning Guide – general Spark performance best practices

Last updated on Nov 20, 2025

Was this page helpful?

Gluten with Velox

Option A: Via spark-submit

Option B: Via spark-defaults.conf

Option C: Programmatic Configuration (Scala/Java)

Option D: PySpark

For JDK 11

For JDK 8

For JDK 11

For JDK 8

Option A: Via `spark-submit`

Option B: Via `spark-defaults.conf`