How to create, track, and pull job results using Dremio's API

Summary

This guide demonstrates how to start, track, and view the results of a Dremio job using only Dremio's API.

Overview

This guide assumes familiarity with the authentication method used by Dremio's API. A link to the authentication documentation is provided in the "Additional Resources" section.

Relevant Versions Tools and Integrations

All versions of Dremio leveraging Dremio's published ("v3") API.

Steps to Resolve

1. Submit an SQL query using the SQL API endpoint, which will return a Job ID.
2. Check the status of the job by passing the Job ID to the Jobs API endpoint. Poll this endpoint until the job status is "COMPLETED".
3. Once the job is completed, retrieve the job results by passing the Job ID to the Job Results API endpoint.

Common Challenges

Your API SQL query may return a limited number of rows due to a configurable batch size set within your Dremio environment. See the "Why does my Rest API SQL query return only 100 rows?" article in the "Additional Resources" section for more information.

Additional Resources

• Why does my Rest API SQL query return only 100 rows?
• Dremio API Authentication Documentation
• Dremio API SQL Documentation
• Dremio API Jobs Documentation
• Dremio API Job Results Documentation