Setting Hadoop Service Log Levels at Runtime

You can dynamically change the log levels of Hadoop services (such as YARN and HDFS) using the hadoop daemonlog command without needing to restart the services.

Key Notes

Ports: Replace <host:port> with the admin/HTTP port of the service (not the web UI port).
Class Names: Use the fully qualified class name of the service’s main class (e.g., org.apache.hadoop.yarn.server.resourcemanager.ResourceManager).
Log Levels: Supported levels: TRACE, DEBUG, INFO, WARN, ERROR, FATAL.
Ensure that the mentioned commands on this page are executed by the user running the respective process. For NameNode-related operations, specifically use the NN service keytab and the hdfs user to initialize the Kerberos token before running the command.
Persistence: Changes are temporary and reset after service restart. For permanent changes, modify log4j.properties and restart the service.

YARN Services

Resource Manager

Get Current Log Level

Bash
    
 
hadoop daemonlog -getlevel <resourcemanager_host>:8088 \\  org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
Copy

The example output is as follows.

Bash
    
 
Connecting to <http://rm-host:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.ResourceManagerSubmitted Class Name: org.apache.hadoop.yarn.server.resourcemanager.ResourceManagerLog Class: org.apache.commons.logging.impl.Log4JLoggerEffective Level: INFO
Copy

Set Log Level to Debug

Bash
    
 
hadoop daemonlog -setlevel <resourcemanager_host>:8088 \\  org.apache.hadoop.yarn.server.resourcemanager.ResourceManager DEBUG
Copy

The example output is as follows.

Bash
    
Connecting to http://rm-host:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.ResourceManager&level=DEBUGSubmitted Class Name: org.apache.hadoop.yarn.server.resourcemanager.ResourceManagerLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: DEBUGSetting Level to DEBUG ...Effective Level: DEBUG
Copy

Node Manager

Get Current Log Level

Bash
    
 
hadoop daemonlog -getlevel <nodemanager_host>:8042 \\  org.apache.hadoop.yarn.server.nodemanager.NodeManager
Copy

The example output is as follows.

Bash
    
 
Connecting to http://trial04.acceldata.dvl:8042/logLevel?log=org.apache.hadoop.yarn.server.nodemanager.NodeManagerSubmitted Class Name: org.apache.hadoop.yarn.server.nodemanager.NodeManagerLog Class: org.apache.commons.logging
Copy

Set Log Level to WARN

Bash
    
 
hadoop daemonlog -setlevel <nodemanager_host>:8042 \\  org.apache.hadoop.yarn.server.nodemanager.NodeManager WARN
Copy

The example output is as follows.

Bash
    
 
Connecting to http://NM-Host:8042/logLevel?log=org.apache.hadoop.yarn.server.nodemanager.NodeManager&level=WARNSubmitted Class Name: org.apache.hadoop.yarn.server.nodemanager.NodeManagerLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: WARNSetting Level to WARN ...Effective Level: WARN
Copy

HDFS Services

NameNode

Get Log Level

Bash
    
 
hadoop daemonlog -getlevel <namenode_host>:50070 \\  org.apache.hadoop.hdfs.server.namenode.NameNode
Copy

The example output is as follows.

Bash
    
 
Connecting to http://NN-HOST:50070/logLevel?log=org.apache.hadoop.hdfs.server.namenode.NameNodeSubmitted Class Name: org.apache.hadoop.hdfs.server.namenode.NameNodeLog Class: org.apache.commons.logging.impl.Log4JLoggerEffective Level: INFO
Copy

Set Log Level to Error

Bash
    
 
hadoop daemonlog -setlevel <namenode_host>:50070 \\  org.apache.hadoop.hdfs.server.namenode.NameNode ERROR
Copy

The example output is as follows.

Bash
    
 
Connecting to http://NN-HOST:50070/logLevel?log=org.apache.hadoop.hdfs.server.namenode.NameNode&level=ERRORSubmitted Class Name: org.apache.hadoop.hdfs.server.namenode.NameNodeLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: ERRORSetting Level to ERROR ...Effective Level: ERROR
Copy

DataNode

Get Log Level

Bash
    
 
hadoop daemonlog -getlevel <datanode_host>:50075 \\  org.apache.hadoop.hdfs.server.datanode.DataNode
Copy

The example output is as follows.

Bash
    
 
Connecting to http://DN-HOST:50075/logLevel?log=org.apache.hadoop.hdfs.server.datanode.DataNodeSubmitted Class Name: org.apache.hadoop.hdfs.server.datanode.DataNodeLog Class: org.apache.commons.logging.impl.Log4JLoggerEffective Level: INFO
Copy

Set Log Level to Debug

Bash
    
 
hadoop daemonlog -setlevel <datanode_host>:50075 \\  org.apache.hadoop.hdfs.server.datanode.DataNode DEBUG
Copy

The example output is as follows.

Bash
    
 
Connecting to http://DN-HOST:50075/logLevel?log=org.apache.hadoop.hdfs.server.datanode.DataNode&level=DEBUGSubmitted Class Name: org.apache.hadoop.hdfs.server.datanode.DataNodeLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: DEBUGSetting Level to DEBUG ...Effective Level: DEBUG
Copy

Other Services

MapReduce JobHistory Server

Set Log Level

Bash
    
 
hadoop daemonlog -setlevel <historyserver_host>:19888 \\  org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer DEBUG
Copy

The example output is as follows.

Bash
    
 
Connecting to http://MR-HOST:19888/logLevel?log=org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer&level=DEBUGSubmitted Class Name: org.apache.hadoop.mapreduce.v2.hs.JobHistoryServerLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: DEBUGSetting Level to DEBUG ...Effective Level: DEBUG
Copy

Timeline Server

Set Log Level

Bash
    
 
hadoop daemonlog -setlevel <timelineserver_host>:8188 \\  org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer WARN
Copy

The example output is as follows.

Bash
    
Connecting to http://Timeline-Host:8188/logLevel?log=org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer&level=WARNSubmitted Class Name: org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServerLog Class: org.apache.commons.logging.impl.Log4JLoggerSubmitted Level: WARNSetting Level to WARN ...Effective Level: WARN
Copy

Verification

Check logs in real-time

Bash
    
 
tail -f /var/log/hadoop-yarn/yarn-*-resourcemanager-*.log
Copy

Confirm log level updates

Bash
    
 
hadoop daemonlog -getlevel <host:port> <classname>
Copy

Example Summary

Service	Command (Set to Debug)
ResourceManager	`hadoop daemonlog -setlevel rm-host:8088 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager DEBUG`
NodeManager	`hadoop daemonlog -setlevel nm-host:8042 org.apache.hadoop.yarn.server.nodemanager.NodeManager DEBUG`
NameNode	`hadoop daemonlog -setlevel nn-host:9870 org.apache.hadoop.hdfs.server.namenode.NameNode DEBUG`

For HTTPS clusters, add -protocol https to the command.

Last updated on Jun 17, 2025

Was this page helpful?