Summary/Reported Issue
User experienced metadata refresh issues in their MongoDB source which caused columns not to be in sync and missing columns from tables.
Relevant Versions
Dremio 24.3.x
Troubleshooting Steps
- We were able to identify that the metadata refresh issues were caused by a lack of privileges for the `dbStats` function for the user used to authenticate in Dremio. The relevant error message can be seen below:
2025-03-24 12:17:39,874 [metadata-refresh-modifiable-scheduler-14] ERROR c.d.p.mongo.MongoDocumentHelper - Failed to run command `Document{{dbStats=1}}`, exception : com.mongodb.MongoCommandException: Command failed with error 13 (Unauthorized): 'not authorized on admin to execute command { dbStats: 1, $db: "admin", $clusterTime: { clusterTime: Timestamp(1742818659, 180), signature: { hash: BinData(0, C4ECD0C94B3C355DBDD09FAF9508E7BD75409EB1), keyId: 7425171175363313665 } }, lsid: { id: UUID("e01297c0-aa98-41f5-a921-a6b196441807") } }' on server pl-0-us-east-1.xwvr1.mongodb.net:1027. The full response is {"ok": 0.0, "errmsg": "not authorized on admin to execute command { dbStats: 1, $db: \"admin\", $clusterTime: { clusterTime: Timestamp(1742818659, 180), signature: { hash: BinData(0, C4ECD0C94B3C355DBDD09FAF9508E7BD75409EB1), keyId: 7425171175363313665 } }, lsid: { id: UUID(\"e01297c0-aa98-41f5-a921-a6b196441807\") } }", "code": 13, "codeName": "Unauthorized", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1742818659, "i": 180}}, "signature": {"hash": {"$binary": {"base64": "xOzQyUs8NV290J+vlQjnvXVAnrE=", "subType": "00"}}, "keyId": 7425171175363313665}}, "operationTime": {"$timestamp": {"t": 1742818659, "i": 180}}}
- We were able to see that the metadata problems went away once we provided access to this function.
Cause
Dremio requires access to dbStats in order to get table statistics. Not having access to this function will cause metadata refreshes to fail.
Steps to Resolve
- DBA assigned `dbStats` function which corrected metadata refresh issues.