Summary
On earlier versions of Dremio with Iceberg enabled for metadata, users may see errors linked to commits to Iceberg
Reported Issue
The user may see the following error in the UI
Also in the server.log on the coordinator we see the following
2022-02-23 16:54:34,040 [e1 - 1de99947-25a7-22fa-79cf-aa7a90167000:frag:0:0] ERROR com.dremio.sabot.driver.SmartOp - NessieReferenceConflictException: Retry-Failure during commit against 'main@a9109d7fc12e1987ba3b73ea46cad8fb01d2f6d526ead500514884f6df1105cb'
... (error log omitted)
Relevant Versions
Dremio Versions earlier than 21.0
Troubleshooting Steps
N/A
Cause
When the underlying metadata in Dremio is managed by Iceberg, updates and changes are made by committing new versions to Iceberg. If client sessions (UI users, ODBC tools etc) have outdated versions of the Iceberg schema then these errors can occur. Internal jira DX-44953 covers this scenario and the work done to correct it.
Steps to Resolve
In Dremio versions up to and including 20.0, users can increase the value of the support key nessie.kvversionstore.max_retries from its default of 10 to a higher value for enhanced support.
Starting with Dremio version 21.0, this support key is replaced by nessie.kvversionstore.commit_timeout_ms.
Tips & Tricks
N/A
Best Practices
N/A
Recommendations
N/A
FAQ
N/A
Additional Resources
https://docs.dremio.com/software/release-notes/210-release/#issues-fixed