Summary
Using ADSLv2 as distributed storage for Dremio without enabling "Hierarchical namespace" can result in the error This endpoint does not support BlobStorageEvents or SoftDelete
Reported Issue
The following error can be returned in the Dremio coordinators server.log and in the Dremio UI when using Azure Data Lake Storage Gen2 for the pdfs configuration in dremio.conf
source ["__home"] is currently unavailable. Metadata is not accessible;
please check node health (or external storage) and permissions.
Info: [Operation failed:
"This endpoint does not support BlobStorageEvents or SoftDelete.
Please disable these account features if you would like to use this endpoint."Relevant Versions
All Dremio Releases.
Cause
This error is caused when the ADLSv2 storage account is configured with a combination of Blob soft delete Enabled and Hierarchical namespace Disabled like so:
Azure Portal -> Storage account -> Overview -> Properties
Steps to Resolve
Enable the Hierarchical namespace property. To do this click on the existing property value (Disabled) in Azure. This will take you to the the Data Lake Gen2 upgrade page. Follow the steps presented to you to complete the change. Microsoft documentation for this process can be found in the Additional Resources section of this article.
Additional Resources
Upgrade Azure Blob Storage with Azure Data Lake Storage Gen2 capabilities