How to connect to Dremio Cloud from Soda Core – Dremio Support

Summary

Soda Core is a data quality tool that integrates with various data sources. While the official documentation supports connecting to Dremio Software, connecting Soda Core to Dremio Cloud is not explicitly documented. However, with some configuration adjustmentsm you can successfully established this connection using Personal Access Tokens (PATs). This article provides guidance and example configurations for connecting Soda Core to Dremio Cloud, as well as troubleshooting tips and caveats.

Main Content

Prerequisites

Soda Core installed: Ensure you have a working installation of Soda Core.
Personal Access Token (PAT) from Dremio Cloud: Create a PAT as described in the Dremio Cloud documentation.
pyodbc Python library: Make sure this is installed (pip install pyodbc).

Example Configuration

Below is a working configuration for connecting Soda Core to Dremio Cloud. Replace placeholders with your specific credentials and values.

data_source dremio_cloud:
  type: dremio
  host: data.eu.dremio.cloud     # Or data.dremio.cloud for US region
  port: "443"
  username: ""                   # Leave blank for PAT authentication
  password: ""                   # Leave blank for PAT authentication
  token: ${PAT_TOKEN}            # Substitute with your actual PAT or environment variable
  schema:
  use_encryption: "true"
  disable_certificate_verification: "true"

Tips:

token should be set to your Dremio PAT. You can use environment variables for better security.
use_encryption and disable_certificate_verification are recommended for encrypted cloud connections. Adjust as necessary for your security posture.
Leave username and password blank when using token authentication.

Setting Environment Variable for PAT

For Unix/Mac:

export PAT_TOKEN="your-personal-access-token-here"

For Windows:

set PAT_TOKEN=your-personal-access-token-here

Troubleshooting Steps

Error: “Unable to connect”
- Check that your PAT is valid and not expired.
- Ensure the host is set to data.eu.dremio.cloud (or data.dremio.cloud for US region).
- Confirm port is 443.
Error: “Authentication failed”
- Verify that username and password are empty.
- Confirm that token is formatted and populated correctly.

FAQ

Q: Where do I find my Dremio Cloud Personal Access Token?

A: Generate it from the Account Settings > Security section in Dremio Cloud.

Q: Will future versions of Soda Core support Dremio Cloud natively?

A: This is unknown; contact Soda for more information.

Supplementary Information

More on Soda Dremio configuration: Soda Data Source Reference