Summary
This article highlights the necessary logs for identifying the root cause after an unexpected restart and guides how to collect them.
Reported Issue
Unexpected Restart of Dremio
Overview
This article covers how to collect information for Dremio Support to support an RCA following an unexpected process restart
Relevant Versions Tools and Integrations
All versions of Dremio and deployment types.
Steps to Resolve
N/A
Collecting Dremio logs
Using DDC
The easiest way to collect Dremio logs is by using the Dremio Diagnostic Collector (DDC). The tool will collect logs and config files from Dremio clusters for all deployment types using either ssh access or kubetctl as appropriate.
See the following article:
https://support.dremio.com/hc/en-us/articles/15560006579739
The tool does not need any build or install, although you can if you wish to compile from source. The install instructions in the above link are simply example commands to obtain the binary.
Using manual methods
If the DDC is not wanted, then the files can be collected manually or with a shell script or other tool. Bear in mind the time and date of the incident and the files that are collected. It might seem obvious, but it is very important to keep the time frame of the incident in mind when collecting log files to make sure the correct ones are sent.
Collecting OS info
Often it is useful to collect some OS level info, especially in the case of an unplanned process restart. If a process was killed by the OS, for example, this will not usually appear in application logs. However, we would likely see this in dmesg or syslog output.
dmesg -T > $(hostname).dmesg.out
Depending on the OS, syslog or messages are usually in /var/log . As noted above, the time of the incident dictates which files to collect and provide to us.
Common Challenges
N/A
Additional Resources
https://support.dremio.com/hc/en-us/articles/7296581582235
https://man7.org/linux/man-pages/man1/dmesg.1.html
https://dremio.zendesk.com/knowledge/articles/9726914805787/en-us?brand_id=1260802262170