Overview
Dremio logs information to multiple log files. This article discusses why Dremio may stop logging information as expected.
Applies To
Dremio Software releases that use the logback framework to configure logging.
Details
The information in a Dremio log file, such as server.log, stops updating, and the file modification time in an "ls -l" output in the log directory is no longer current.
Cause
Dremio uses logback to define and control what is logged by the server and its components. The default logback configuration file for server.log is conf/logback.xml.
If the contents of the logback.xml have been incorrectly updated or modified, logging to the server.log may stop following any future restart of the Dremio process. No error or warning will be displayed to advise the user of the error.
If dynamic logging has been configured in logback.xml the server.log will stop logging once the scanPeriod has elapsed. Again, no error or warning will be displayed to advise the user of the error.
As an example, if you add a new logger to logback.xml to track debug messages within the Metadata code, a correct entry in the conf/logback.xml would look like:
<logger name="com.dremio.exec.catalog.MetadataSynchronizer"additivity="false"><level value="debug"/><appender-ref ref="text"/>
</logger>
If, for example, a simple spelling mistake was made in the word "logger" so that it read "lgger" , like so
<lgger name="com.dremio.exec.catalog.MetadataSynchronizer"additivity="false"><level value="debug"/><appender-ref ref="text"/>
</logger>
... then all logging to the server.log would stop following a restart or when scanPeriod has elapsed if dynamic logging is configured.
Solution
Revert any changes made to the logback.xml and restart the Dremio process, or allow the scanPeriod to elapse if dynamic logging is used.
It is advisable to create a copy of the logback.xml file before any changes are introduced so that the change can be easily compared to the original and reverted if needed.
Further Reading
Logback Documentation: https://logback.qos.ch/documentation.html
Dynamic Logging: https://support.dremio.com/hc/en-us/articles/9726914805787-How-to-enable-dynamic-logging-in-Dremio
Dremio Logs : https://docs.dremio.com/software/advanced-administration/log-files