Allowing use of KMS Keys in Dremio Cloud for the Project Store

Summary

This article provides steps to allow the Dremio Cloud project store role to use a KMS key that is enforced in your organization.

Overview

Your company may have EBS Auto Encryption enabled in their AWS organizations. This will enforce the encryption of the new EBS volumes. If you do not attach the proper KMS policies and keys to the project store role, the EC2 instances that Dremio Cloud spins up will fail and automatically terminate due to being unable call the KMS actions, or error out with a kms:decrypt error when querying a table.

Relevant Versions Tools and Integrations

Dremio Cloud

Steps to Resolve

1. Make note of the default encryption key in the EBS encryption settings (found in AWS EC2 -> Settings). Your project store would have been auto encrypted with this key if it is on by default.

   Open Screenshot 2023-11-08 at 4.10.44 PM.png

   

2. Navigate to the Project Store’s S3 bucket in AWS and select properties

   Open Screenshot 2023-11-08 at 4.38.16 PM.png

   

3. Scroll down and check if the KMS key matches from step 1

   Open Screenshot 2023-11-08 at 4.39.18 PM.png

   

4. Navigate to the AWS Key Management Service console

5. Select the default key used for encryption and select the Key policy tab

   Open Screenshot 2023-11-08 at 4.14.08 PM.png

   

6. Select Edit and add or edit the below policy. Specify the project store role arn
   
   

   { "Sid": "Allow role the use of KMS key", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<account-id):role/cindy-dcs-DremioIAMProjectDataAccess-role", ] }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey", "kms:CreateGrant", "kms:Decrypt", "kms:DescribeKey", "kms:GenerateDataKeyWithoutPlainText", "kms:ReEncrypt" ], "Resource": "*" }
   Open Screenshot 2023-11-08 at 4.18.01 PM.png

   

7. Save the changes

8. Navigate to the Project Store IAM role in AWS (xxxxxx-DremioIAMProjectDataAccessRole)

9. Select “Add Permissions”

10. Select Create Inline Policy and name it

11. Copy and paste this below policy, with the default KMS key

    { "Action": [ "kms:*" ], "Effect": "Allow", "Resource": [ "arn:aws:kms:us-east-1:5<account-id>:key/<key>" ], "Sid": "Stmt1554423050001" } ] }

12. Save the policy

13. Spin up engines in Dremio Cloud by running a query, if it succeeds, then the correct policies have been applied properly. If it fails after a few minutes, the wrong key may be specified and the instance will shut down on the AWS side.