Troubleshoot Using HDFS Audit Logs

Pulse enables you to view, search, and analyze HDFS audit logs to monitor file system activity and ensure compliance.

Access HDFS Audit Logs

1. In the Pulse UI, select Logs from the left navigation bar.
2. Click the HDFS Audit tab.
3. Set the time range – select Today, Last 12 Hours, Last 3 Months, or define a custom period, then click Apply.

The HDFS Audit logs appear on the screen.

Features and Functionality

Search Logs

On the Logs page > HDFS Audit page, select a search type:

• Standard Search: Use filters such as UGI, CMS, SRC, DST, CMD, PERM, Allowed, IP, Proto, Caller Context, Host, Log Level, and Service.
• Elastic QS Search: Use Elastic Query Syntax (QS) to create search queries.

In the search bar, enter your query, and Pulse displays the matching logs within the selected time range.

Best practices:

• Use exact values for accurate results.
• Combine multiple parameters to refine your search.
• For guidelines and examples of standard search queries, see Search and Analyze Records
• For examples of Elastic search queries, see Sample Elasticsearch Queries for Searching Logs.

Filter Logs

On the Logs > HDFS Audit page, apply filters to narrow results:

• Service, Host, or Log Level: Focus on specific components.
• Command (CMD): Filter logs by executed commands.
• Protocol (Proto): Filter by network protocol.
• Allowed: Select True or False to see allowed or denied operations.
• Time Range: Choose a preset range or define a custom period.
• Multiple Filters: Apply multiple filters simultaneously.

Filtered logs update automatically in histograms, dashboards, and log details panels.

Visual Log Insights

On the Logs > HDFS Audit page, Pulse provides an interactive visualization to analyze trends:

• Time histograms – Display the number of logs over time.

Visualizations update automatically as you apply filters or search queries, ensuring real-time visibility into your cluster’s activity.

Detailed Log Messages

On the Logs > HDFS Audit page, review full details of HDFS audit events:

• Message Details: View log time, UGI, command, source path, destination path, permissions, allowed status, IP address, protocol, caller context, service, and host.
• View in Context: See related logs before and after a selected event for full context.
• Copy Messages: Copy log details to the clipboard for reporting, offline analysis, or troubleshooting.
• Search messages – Enter keywords to quickly find specific log messages.

Detailed messages update automatically as you apply filters or search queries, helping you troubleshoot issues efficiently.

For more details about the Log Messages, see HDFS Audit Logs.

Group and Export Logs

On the Logs > HDFS Audit page, organize and export HDFS audit logs for structured analysis:

• Group Logs: Group by Trace, “Level or Severity”, or Host to analyze patterns efficiently.
• Export Logs: Download in .xlsx or .logs format. You can specify the number of rows to download.

Grouped and exported logs reflect applied filters and search queries, ensuring relevant data is captured.