FE Log Management
This document mainly introduces the log management of the Frontend (FE) process.
This document is applicable to Doris versions 2.1.4 and later.
Log Categories
When starting the FE process using sh bin/start_fe.sh --daemon
, the following types of log files will be generated in the FE log directory:
fe.log
FE process running log. The main log file for FE. All levels of FE process logs (DEBUG, INFO, WARN, ERROR, etc.) will be printed to this log file.
fe.warn.log
FE process running log. Only prints logs at WARN level and above. The content in fe.warn.log is a subset of the fe.log log content. It is mainly used to quickly view warning or error level logs.
fe.audit.log
Audit log. Used to record all database operation records executed through this FE node. This includes SQL, DDL, DML statements, etc.
fe.out
Used to receive standard output stream and error data stream logs. For example, output from
echo
commands in start scripts, or other log information not captured by the log4j framework. Usually serves as a supplement to the running log. In rare cases, you may need to check the content of fe.out for more information.fe.gc.log
GC log of the FE JVM. The behavior of this log is controlled by the JVM startup option
JAVA_OPTS
in fe.conf.
Log Configuration
Includes configuring the log storage path, retention time, retention count, size, etc.
The following configuration items are configured in the fe.conf
file.
Configuration Item | Default Value | Options | Description |
---|---|---|---|
LOG_DIR | ENV(DORIS_HOME)/log | Storage path for all logs. By default, it is the log/ directory under the FE deployment path. Note that this is an environment variable, and the configuration name should be in uppercase. | |
sys_log_level | INFO | INFO , WARN , ERROR , FATAL | Log level of fe.log . Default is INFO. Not recommended to change, as INFO level contains many critical log information. |
sys_log_roll_num | 10 | Controls the maximum number of files for fe.log and fe.warn.log in one day. Default is 10. When the number of log files exceeds this threshold due to log rolling or splitting, older log files will be deleted. | |
sys_log_verbose_modules | Can set specific Java package files to enable DEBUG level logging. See the "Enable DEBUG Log" section for details. | ||
sys_log_enable_compress | false | true, false | Whether to enable compression for historical fe.log and fe.warn.log logs. Default is off. When enabled, historical audit logs will be archived using gzip compression. |
log_rollover_strategy | age | age , size | Log retention strategy, default is age , which retains historical logs based on time. size retains historical logs based on log size. |
sys_log_delete_age | 7d | Supports formats like 7d, 10h, 60m, 120s | Only effective when log_rollover_strategy is age . Controls the number of days to retain fe.log and fe.warn.log files. Default is 7 days. Logs older than 7 days will be automatically deleted. |
audit_log_delete_age | 7d | Supports formats like 7d, 10h, 60m, 120s | Only effective when log_rollover_strategy is age . Controls the number of days to retain fe.audit.log files. Default is 30 days. Logs older than 30 days will be automatically deleted. |
info_sys_accumulated_file_size | 4 | Only effective when log_rollover_strategy is size . Controls the cumulative size of fe.log files. Default is 4GB. When the cumulative log size exceeds this threshold, historical log files will be deleted. | |
warn_sys_accumulated_file_size | 2 | Only effective when log_rollover_strategy is size . Controls the cumulative size of fe.warn.log files. Default is 2GB. When the cumulative log size exceeds this threshold, historical log files will be deleted. | |
audit_sys_accumulated_file_size | 4 | Only effective when log_rollover_strategy is size . Controls the cumulative size of fe.audit.log files. Default is 4GB. When the cumulative log size exceeds this threshold, historical log files will be deleted. | |
log_roll_size_mb | 1024 | Controls the maximum size of individual files for fe.log , fe.warn.log , fe.audit.log . Default is 1024MB. When a single log file exceeds this threshold, a new file will be created automatically. | |
sys_log_roll_interval | DAY | DAY , HOUR | Controls the rolling interval of fe.log and fe.warn.log . Default is 1 day, generating a new log file every day. |
audit_log_roll_num | 90 | Controls the maximum number of files for fe.audit.log . Default is 90. When the number of log files exceeds this threshold due to log rolling or splitting, older log files will be deleted. | |
audit_log_roll_interval | DAY | DAY , HOUR | Controls the rolling interval of fe.audit.log . Default is 1 day, generating a new log file every day. |
audit_log_dir | ENV(DORIS_HOME)/log | Can specify a separate storage path for fe.audit.log . Default is the log/ directory under the FE deployment path. | |
audit_log_modules | {"slow_query", "query", "load", "stream_load"} | Module types in fe.audit.log . Default includes slow query, query, load, stream load. "Query" includes all DDL, DML, SQL operations. "Slow query" refers to operations that exceed the qe_slow_log_ms threshold. "Load" refers to Broker Load. "Stream load" refers to stream load operations. | |
qe_slow_log_ms | 5000 | When the execution time of DDL, DML, SQL statements exceeds this threshold, it will be separately recorded in the slow_query module of fe.audit.log . Default is 5000 ms. | |
audit_log_enable_compress | false | true, false | Whether to enable compression for historical fe.audit.log logs. Default is off. When enabled, historical audit logs will be archived using gzip compression. |
sys_log_mode | NORMAL | NORMAL , BRIEF , ASYNC | FE log output mode, where NORMAL is the default output mode, log output is synchronous and includes location information. ASYNC is the default log output is asynchronous and includes location information. BRIEF mode is log output asynchronously but does not include location information. The performance of the three log output modes increases in sequence. |
::: note
Starting from version 3.0.2, the default value of sys_log_mode
configuration is changed to AYSNC
.
:::
sys_log_roll_num
控制的是一天的保留日志数量,而不是总数量,需要配合 sys_log_delete_age
共同确定总保留日志数量。
sys_log_roll_num
controls the number of retained logs per day, not the total number. It needs to be combined with sys_log_delete_age
to determine the total number of retained logs.
Enable DEBUG Log
The Debug level log of FE can be enabled by modifying the configuration file or through the interface or API during runtime.
Enable through configuration file
Add the configuration item
sys_log_verbose_modules
in fe.conf. For example:# Only open DEBUG log for "org.apache.doris.catalog.Catalog"
sys_log_verbose_modules=org.apache.doris.catalog.Catalog
# Open DEBUG log for all classes in "org.apache.doris.catalog"
sys_log_verbose_modules=org.apache.doris.catalog
# Open DEBUG log for all classes in "org"
sys_log_verbose_modules=orgAdd the configuration item and restart the FE node to take effect.
Enable through FE UI interface
You can modify the log level at runtime through the UI interface. No need to restart the FE node. Open the FE node's http port in the browser (default is 8030) and log in to the UI interface. Then click on the
Log
tab in the top navigation bar.In the Add input box, you can enter the package name or specific class name to open the corresponding Debug log. For example, entering
org.apache.doris.catalog.Catalog
will open the Debug log of the Catalog class:You can also enter the package name or specific class name in the Delete input box to close the corresponding Debug log.
:::note The modification here will only affect the log level of the corresponding FE node. It will not affect the log level of other FE nodes. :::
Modify through API
You can also modify the log level at runtime through the following API. No need to restart the FE node.
curl -X POST -uuser:passwd fe_host:http_port/rest/v1/log?add_verbose=org.apache.doris.catalog.Catalog
Where the username and password are the root or admin users logged into Doris. The
add_verbose
parameter specifies the package name or class name for enabling Debug log. If successful, it will return:{
"msg": "success",
"code": 0,
"data": {
"LogConfiguration": {
"VerboseNames": "org,org.apache.doris.catalog.Catalog",
"AuditNames": "slow_query,query,load",
"Level": "INFO"
}
},
"count": 0
}You can also close Debug log through the following API:
curl -X POST -uuser:passwd fe_host:http_port/rest/v1/log?del_verbose=org.apache.doris.catalog.Catalog
The
del_verbose
parameter specifies the package name or class name for closing Debug log.
Log Configuration for k8s
In some cases, the FE process is deployed through a container environment (such as k8s). All logs need to be output through standard output stream instead of files.
In this case, you can start the FE process in the foreground and output all logs to the standard output stream by using the command sh bin/start_fe.sh --console
.
To distinguish different types of logs in the same standard output stream, a different prefix will be added before each log. For example:
RuntimeLogger 2024-06-24 00:05:21,522 INFO (main|1) [DorisFE.start():158] Doris FE starting...
RuntimeLogger 2024-06-24 00:05:21,530 INFO (main|1) [FrontendOptions.analyzePriorityCidrs():194] configured prior_cidrs value: 172.20.32.136/24
RuntimeLogger 2024-06-24 00:05:21,535 INFO (main|1) [FrontendOptions.initAddrUseIp():101] local address: /172.20.32.136.
RuntimeLogger 2024-06-24 00:05:21,740 INFO (main|1) [ConsistencyChecker.initWorkTime():106] consistency checker will work from 23:00 to 23:00
RuntimeLogger 2024-06-24 00:05:21,889 ERROR (main|1) [Util.report():128] SLF4J: Class path contains multiple SLF4J bindings.
The meanings of different prefixes are as follows:
StdoutLogger
: Logs in the standard output stream, corresponding to the content infe.out
.StderrLogger
: Logs in the standard error stream, corresponding to the content infe.out
.RuntimeLogger
: Logs infe.log
.AuditLogger
: Logs infe.audit.log
.- No prefix: Logs in
fe.gc.log
.
Additionally, there is an additional configuration parameter for the container environment:
Configuration Item | Default Value | Options | Description |
---|---|---|---|
enable_file_logger | true | true, false | Whether to enable file logging. Default is true . When starting the FE process with the --console command, logs will be output to both the standard output stream and the normal log file. When set to false , logs will only be output to the standard output stream and will not generate log files. |