How to enable dynamic logging in Dremio

Overview

When working with Dremio Support, we may ask you to collect debug logs. With logback, we can include a scan parameter to allow changes "on the fly" and avoid a restart of the Dremio application, which is not always possible in production environments.

Applies To

All versions of Dremio

Details

With installation types such as AWSE and on-prem, we can change logback.xmlto include the scan parameter. More specifically, changing:

<configuration>

to:

<configuration scan="true" scanPeriod="30 seconds"> 

The next step is to restart the Dremio process so this change will be applied to the running application. Any subsequent changes to thelogback.xml, like the addition of debug logging, will now be scanned and replaced everyscanPeriod.

With Kubernetes based deployments, you first need to edit logback.xml under the<HELM CHART HOME>/configdirectory. Then you will need to redeploy the Helm charts, at which point the Dremio pods will restart (you should therefore schedule for this brief outage).

After making this update, if you need to change the logging in the future all you need to do is edit thedremio-configconfig map on the Kubernetes cluster. For example:

kubectl edit configmap dremio-config

logback.xml: |
<?xml version="1.0" encoding="UTF-8" ?>
<!--
...
<configuration scan="true" scanPeriod="30 seconds">

    <root>
    <level value="${dremio.log.root.level:-info}"/>
    <if condition='isDefined("dremio.log.path")'>
      <then>
        <appender-ref ref="text"/>
        <appender-ref ref="json"/>
      </then>
      <else>
        <appender-ref ref="console"/>
      </else>
    </if>
  </root>

"/private/var/folders/mn/qjh1d6d52hv5k5x8274lz46m0000gn/T/kubectl-edit-1153192427.yaml" 488L, 19246B written
configmap/dremio-config edited

Further Reading

Logback auto scan -https://logback.qos.ch/manual/configuration.html#autoScan
Config maps -https://kubernetes.io/docs/concepts/configuration/configmap/