Collecting files for Support tickets – Dremio Support

Overview

Often if you need to contact support for issues with Dremio, our Support team may ask you for logs and configuration files. This article outlines how you can effectively reduce time sending different files by using the diagnostic collector tool.

If you are having any problems collecting the right files, you can also open up a ticket with Support.

Applies To

All Dremio versions

Details

Logs and Config files

The diagnostic collector tool is open source and can be accessed here:

https://github.com/dremio/dremio-diagnostic-collector

See the following note on using DDC

https://support.dremio.com/hc/en-us/articles/15560006579739

Profiles

Profiles can be downloaded from the UI and uploaded to the ticket, the docs here show how this is done:

https://docs.dremio.com/software/jobs/overview/#3-download-profile

Note that the time the profile is run and the current logs on the coordinator / executors might not be the same, the profile will likely report in local time.

Note how it is very useful to look at the error in the profile, you might see something like this

SYSTEM ERROR: ChannelClosedException: [FABRIC]: Channel closed /10.245.21.40:45678 <--> /10.245.21.80:39632 (fabric server)

Fragment 7:0

[Error Id: 38cf1186-5827-49fd-97ee-559eb23a87fb on dremio-executor-1.dremio-cluster-pod.dremio.svc.cluster.local:0]

  (com.dremio.exec.rpc.ChannelClosedException) [FABRIC]: Channel closed /10.25.1.1:45678 <--> /10.25.1.2:39632 (fabric server)
    com.dremio.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete():211
    com.dremio.exec.rpc.RpcBus$ChannelClosedHandler.operationComplete():199
    com.dremio.exec.rpc.ReconnectingConnection$CloseHandler.operationComplete():150
    com.dremio.exec.rpc.ReconnectingConnection$CloseHandler.operationComplete():136
    com.dremio.services.fabric.FabricConnection$ProxyCloseHandler.operationComplete():153
    com.dremio.services.fabric.FabricConnection$ProxyCloseHandler.operationComplete():134
    io.netty.util.concurrent.DefaultPromise.notifyListener0():577
    io.netty.util.concurrent.DefaultPromise.notifyListenersNow():551
    io.netty.util.concurrent.DefaultPromise.notifyListeners():490
    io.netty.util.concurrent.DefaultPromise.setValue0():615
    io.netty.util.concurrent.DefaultPromise.setSuccess0():604
    io.netty.util.concurrent.DefaultPromise.trySuccess():104
    io.netty.channel.DefaultChannelPromise.trySuccess():84
    io.netty.channel.AbstractChannel$CloseFuture.setClosed():1158
    io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0():760
    io.netty.channel.AbstractChannel$AbstractUnsafe.close():736
    io.netty.channel.AbstractChannel$AbstractUnsafe.flush0():942
    io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.forceFlush():361
    io.netty.channel.nio.NioEventLoop.processSelectedKey():708
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized():650
    io.netty.channel.nio.NioEventLoop.processSelectedKeys():576
    io.netty.channel.nio.NioEventLoop.run():493
    io.netty.util.concurrent.SingleThreadEventExecutor$4.run():989
    io.netty.util.internal.ThreadExecutorMap$2.run():74
    java.lang.Thread.run():750

The error gives an important clue of the name of the executor

dremio-executor-1.dremio-cluster-pod.dremio.svc.cluster.local

If you're using the diagnostic collector tool, then you can include all or some or your executors

Exceptions

Kubernetes clusters may only log to console, so you may not find all the logs you expect with a default helm install. Contact support for details on setting up logs to a persistent volume

Yarn clusters for executors will typically log to stdout/stderr for the application id on the worker nodes for executor logs.