Summary/Reported Issue
We are getting below error when executing a query in Dremio:
SYSTEM ERROR: IllegalStateException: Individual record size (413792 bytes (404 KiB)) is too large to fit into page size (262144 bytes (256 KiB)). Please increase the support option exec.op.join.spill.page_size accordingly.
Relevant Versions
Dremio 25.2.x
Troubleshooting Steps
Increasing the spill page size using the parameter `exec.op.join.spill.page_size` to a value bigger than the record size should prevent the error from happening. You should be able to change the value of `exec.op.join.spill.page_size` on a session basis.
Steps to Resolve
Run the command:
ALTER SESSION SET exec.op.join.spill.page_size=524288
This would set the size of the spill page to 512 KiB which should be bigger than the record size you're handling. If there are any records bigger than 512 KiB you may still get this error and may need to further increase the value.