Overview
Engine scaling failed with the below error message:
Scaling Failed. We currently do not have sufficient m5d.8xlarge capacity in the Availability Zone you requested (<AZ-1>). Our system will be working on provisioning additional capacity. You can currently get m5d.8xlarge capacity by not specifying an Availability Zone in your request or choosing <AZ-2>, <AZ-3>
Relevant Versions
Dremio Cloud
Cause
This error message comes from the AWS side[1] which indicates ec2 instance (m5d.8xlarge) capacity issues in the requested availability zone (AZ-1).
When multiple subnets across different availability zones are specified in the cloud configuration in dremio cloud[2], engines will be spun up across subnets depending on the capacity.
Hence, if engine scaling fails with capacity issues in one of the availability zones, dremio cloud will retry in the other configured subnet(s) in different AZ(s) to scale up the engine.
Steps to Resolve
Below are the possible ways to resolve this error:
1. Reach out to AWS Support team to increase the ec2 instance capacity in the requested availability zone
OR
2. Edit the cloud settings in dremio cloud to add a new subnet in a different availability zone which has the sufficient capacity for the requested ec2 instance type. To edit a cloud[3]:
- On the Clouds page, hover over the row of the cloud and click
that appears next to the cloud.
- In the Edit Cloud dialog, go to the Network Access section, click Add subnet, enter the new subnet ID(s)
- Click Save.
Additional Resources
[2] https://docs.dremio.com/cloud/sonar/cloud-entities/clouds/#adding-a-cloud
[3] https://docs.dremio.com/cloud/sonar/cloud-entities/clouds/#editing-a-cloud