Summary
In some scenarios, runtime filters compiled to prune Parquet files can fail to clean up completely, leading to a query failure. This was resolved in Dremio 25.1.0.
Reported Issue
The query will fail with a message like this:
SYSTEM ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: (3145728) Allocator(runtimefilter-manager) 0/3145728/3145728/9223372036854775807 (res/actual/peak/limit)
Overview
When Dremio plans to read from a range of Parquet files, a filter object is compiled to optimize the determination of what files should pass the filter or be ignored. Due to a defect tracked in internal Jira DX-90445, this can result in Apache Arrow memory buffers being left without a proper cleanup, which eventually leads to a cancellation of the query.
Relevant Versions, Tools & Integrations
This defect was patched in Dremio 25.1.0 and Dremio 24.3.9 (EOL.) Releases prior to this may see the error.
Steps to Resolve
To prevent this error, cluster operators can change the support key exec.non_partitioned_parquet.enable_runtime_filter from true to false.
Note: This is not a supported configuration, and will cause slower query completion and increased memory consumption, so, should be considered only for critical scenarios.
Next Steps
If this error is encountered, you should plan to upgrade to the latest supported release of Dremio which will include the fix.