Summary
A user might see one of the following errors appear when promoting/formatting a Delta Lake table:
Commit Json for version 0 not found while reading metadata
Failed to read metadata for delta dataset
Reported Issue
Add additional content.
Relevant Versions
Dremio Cloud supports minReaderVersion 2
Dremio Software v25+ supports minReaderVersion 2
Dremio Software prior to v25 only supports minReaderVersion 1
Cause
Dremio does not support the minReaderVersion set in Delta Lake.
Reader Version 3 is not supported by any version of Dremio as of the writing of this article. The feature is tracked internally as PM-2141.
Steps to Resolve
Set "minReaderVersion" in the _delta_log/00000000000000000000.json
files to a value that is supported, either 1 or 2 depending on which Dremio version you are using.
Recommendations
Unity Catalog is Databrick’s proprietary Delta catalog. Many customers have been using our existing Delta table capabilities to interact with underlying delta tables. By connecting to the Unity Catalog, customers will be able to now interact with the centralized catalog of all their Delta tables in the DBX ecosystem. To query these tables, users will need to enable Uniform, which is a iceberg metadata layer which provides an ready-only interoperability layer for iceberg clients.
Additional Resources
https://docs.delta.io/2.0.0/versioning.html
https://docs.dremio.com/current/sonar/query-manage/data-formats/delta-lake/