Summary
Post upgrade of Dremio, users may see errors in the UI while trying to navigate datasets as well as queries waiting to be submitted due to invalid custom ARP connectors / custom plugin JAR files in the /opt/dremio/jars/3rdparty directory of the coordinator. NOTE: Deployment types can also include on-Premise as well as Kubernetes.
Reported Issue
After upgrading Dremio the following error may be seen in the UI or the coordinator logs.
INTERNAL: Encountered end-of-stream mid-frameTypical symptoms involve the UI being slow as well not being able to run queries where Dremio UI displays "Waiting to be submitted" and/or a shows a blank display for Setting > Queues.
Relevant Versions
This issue applies to Dremio versions 25.x and on wards.
Troubleshooting Steps
1. Start a Google Chrome network trace via Developer tools and attempt to run a query
2. Confirm the error seen is the same as the below illustration:
3. Navigate to the coordinator server log and also confirm if the following error can be seen:
2025-08-01 09:41:44,513 [qtp1092986520-1562] ERROR c.d.d.server.GenericExceptionMapper - Unexpected exception when processing GET http://localhost:9047/apiv2/job/177374f4-362a-5516-4226-bf84d4bd7a00/data?offset=0&limit=100 : org.apache.arrow.flight.FlightRuntimeException: INTERNAL: Encountered end-of-stream mid-frame
org.apache.arrow.flight.FlightRuntimeException: Encountered end-of-stream mid-frame
at org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:131)
at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:165)
at org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:186)
at org.apache.arrow.flight.FlightStream$Observer.onError(FlightStream.java:467)
at org.apache.arrow.flight.FlightClient$1.onError(FlightClient.java:351)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at org.apache.arrow.flight.grpc.ClientInterceptorAdapter$FlightClientCallListener.onClose(ClientInterceptorAdapter.java:116)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.opentracing.contrib.grpc.TracingClientInterceptor$1$2.onClose(TracingClientInterceptor.java:224)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:564)
at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:72)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:729)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:710)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
4. Carry out the "Steps to Resolve" described below.
Cause
This is due to one or more invalid custom ARP connectors / custom plugin JAR files in the /opt/dremio/jars/3rdparty directory of the coordinator. This could be because those custom JAR files are outdated and not compatible with the upgraded version of Dremio or clash with an already existing Dremio ARP connector / JAR file of the same filename.
Steps to Resolve
Resolve custom ARP connectors / custom plugin JAR file conflict by either upgrading to a suitable latest version of the JAR or remove if conflicting with an already existing out of the box Dremio ARP connector JAR file.
Additional Resources
Custom ARP Connector: https://docs.dremio.com/current/developer/arp-connector/