Configuring External Token Providers in Dremio with KeyCloak as an IdP

Summary

This is a practical how-to for configuring the External Token Provider functionality in the Dremio UI, with KeyCloak acting as the authorization server / identity provider.

Reported Issue

This article came about from a customer reporting issue where the following error was seen: WARN c.d.s.u.SimpleAccessTokenService - Token authentication failed, error parsing the token: Illegal base64 character 2e when testing External Token Providers with Dremio v24.3.4.

Overview

As of v24.3.0, Dremio Software supports the use of external token providers, so a JSON Web Token (JWT) issued by an OpenID Connect (OIDC)-conformant authorization server can be used to establish ODBC/JDBC (non-Arrow Flight) connections to Dremio. The authorization server can be any JWT provider, including an identity provider (IdP).

In this example we use the open source solution KeyCloak as the authorization server and configure it as the external token provider in the Dremio UI.

NOTE: This example uses a local on-premises instance of Dremio, but the external token provider configuration is relevant for any Dremio deployment type.

Relevant Versions Tools and Integrations

This working example applies to Dremio versions 24.3.x, 25.x and 26.x. Steps were re-verified with KeyCloak 26.8.0.

Steps to Resolve

This article assumes KeyCloak is installed and running with user accounts already added in KeyCloak. Full background on KeyCloak can be found at www.keycloak.org, and https://github.com/keycloak.

1. Start by connecting to the KeyCloak Admin/Dashboard on port 8080. In this example KeyCloak is running locally so we will open http://localhost:8080/admin/master/console/.

2. For the purpose of simplicity we will be using the master realm which is the default when standing up a KeyCloak instance.

3. On KeyCloak Admin/Dashboard select the Clients tab on the navigation bar, and Create client. Leave Client type as OpenID Connect and set Client ID to dremio (any name works, but it must match everywhere it is referenced below; Name and Description can be left blank) and click Next.

4. On the Capability config screen:
- Toggle Client authentication On. This is required for the generation of the Client Secret for the HTTP POST call we will make to get our JWT later on.
- Under Authentication flow, make sure Direct access grants is checked. This is required for the grant_type=password request in Step #14. Without it KeyCloak returns {"error":"unauthorized_client","error_description":"Client not allowed for direct access grants"}.
- Click Next. On the Login settings screen we can leave all fields as default, since in the external token provider model Dremio does not consume the authorization code from the front end call during SSO, so we can just click Save here.

5. This will now take you to the newly created Client and the client details screen

6. From the Client scopes tab of this client (Clients > dremio > Client scopes), click on the <client_name>dedicated scope from the list. The name should match up with the same name given to the created client above. In our case this is dremio-dedicated. 

NOTE: This scope does not appear under the realm-level Client scopes menu item, only inside the client.

7. Then click Configure a new mapper and from the list select the 'Audience' mapping. For the Name field, give the mapper any name and then click Save. In this example we will name this 'dremio'. For the Included Client Audience field, click the drop down and select the client that was previously created. In our case our client is 'dremio' so we will select this. Make sure Add to access token is On. NOTE: Keep a note of this name as we will need this to populate the Audience value for our External Token Provider setup in the Dremio UI. Once configured, the decoded access token will contain "aud": ["dremio", ...].

--------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------

8. Now start a fresh instance of Dremio v24.3.x or later. In our example we are using a local instance of Dremio with no SSO authentication or Wire Encryption enabled so our dremio.conf is a default one as below.

NOTE: This configuration setup for an External Token Provider will also work with existing installations of Dremio.

paths: {
# the local path for dremio to store data.
local: ${DREMIO_HOME}"/data"

# the distributed path Dremio data including job results, downloads, uploads, etc
dist: "pdfs://"${paths.local}"/pdfs"
}

services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true,
flight.use_session_service: true

}

9. Since we have configured a local instance of Dremio with no Identity Provider for authentication, we need to make sure the new user we create in Dremio UI after the EULA page matches the same user we used to login to the KeyCloak Admin/Dashboard. In this example we are using admin/admin as the username/password account for KeyCloak so we will create the dremio first time user as 'admin'. NOTE: If using an identity provider such as LDAP (ad.json)/AzureAD (azuread.json) or even KeyCloak (oauth.json) for user authentication, then the user must match up to the preferred_username value (or any other username mapping claim from KeyCloak) within the JWT issued by KeyCloak.

10. Once the new user is created in the 'Welcome to Dremio', 'Create Admin Account' page, click on the Settings cog icon on the lower left of the Dremio UI navigation bar and select the External Token Providers tab.

11. Now click Add Provider and fill in the following details
- Name: We will use 'keycloak'
- Audience: We will use 'dremio' which is the same name we gave to the Audience mapper on Step #7
- User Claim Mapping: Since we are using KeyCloak, the user claim mapping field that identifies the Dremio username of the JWT user is 'preferred_username'. So this is what we will use.
- Issuer URL: We can get this from the issuer field on the KeyCloak http://<host>:<port>/realms/master/.well-known/openid-configuration URL. In our example this is: 'http://localhost:8080/realms/master'

- JWKS URL (optional): We can leave this blank as Dremio will pick this up automatically

12. Now click Add. NOTE: If any of the above details are incorrect, you will get the following error pop-up: INVALID_ARGUMENT: Could not lookup OpenID configuration for the provided issuer.

13. Assuming all went well adding the provider, if we now edit the provider, we can notice that Dremio has looked-up and added the JWKS URL.

14. The next step is to now get a JWT token. In this example to keep things simple we will be using cURL/Postman to hit the correct KeyCloak endpoint (with the relevant credentials and information in the HTTP POST command) to get our JWT. The client_secret is under Clients > dremio > Credentials tab (if there is no Credentials tab, Client authentication is not enabled, see Step #4).

If using cURL (Linux/macOS/WSL), we issue the below POST command to our local KeyCloak instance:

curl --location 'http://localhost:8080/realms/master/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'client_id=dremio' \
--data-urlencode 'client_secret=<CLIENT_SECRET>' \
--data-urlencode 'username=admin' \
--data-urlencode 'password=<KEYCLOAK_ADMIN_PASSWORD>'

NOTE: On Windows PowerShell, curl is an alias for Invoke-WebRequest and the \ line continuations do not work. Use curl.exe on a single line, run the command from WSL/bash, or use Postman.

NOTE: grant_type=client_credentials can also be used instead depending on the customer environment and setup. In that case enable Service accounts roles on the client, omit username/password, and the resulting username claim will be service-account-dremio, so a Dremio user with that name must exist.

If using Postman, we will issue the below POST request with field values against our local KeyCloak instance. NOTE: We got the client_secret from the Credentials tab of our configured KeyCloak client.

--------------------------------------------------------------------------------------------------------------------

15. Now take a copy of the returned access_token value (not the refresh_token). NOTE: By default KeyCloak access tokens expire after 60 seconds ("expires_in": 60), so use the token promptly or increase Realm settings > Tokens > Access Token Lifespan.

16. Open up a any third-party SQL client such as DBeaver and create a database connection with Dremio using the following JDBC connection details:

- JDBC URL: jdbc:dremio:direct=localhost:31010;token_type=jwt 
- Username: We will leave as blank
- Password: We will copy in the JWT (access_token) we got from step #15.

17. NOTE: We set the property token_type to jwt in our connection string per Step #4. of the Dremio Documentation: https://docs.dremio.com/current/sonar/security/authentication/external-token

18. Now click Test Connection and you should get a Connected message from DBeaver like the below.

19. At the same time the following should be logged in the Dremio coordinator server.log.

2024-05-22 11:19:59,294 [UserServer-1] INFO com.dremio.ConnectionLog - [0160faf1-44cd-4a12-a69c-c76b64d48825] Connection opened.
Endpoint: 127.0.0.1:55376
Protocol Version: 5
Record Type: DREMIO
Record Formats: DREMIO_23_0, DREMIO_1_4, DREMIO_0_9
Support Complex Types: true
Name: Dremio JDBC Driver
Version: 24.2.2-202309281602050954-7d5e877f (24.2.2)
Application: 14560@LAPTOP-MJMBCBJM
User Properties:
useEncryption=false
direct=localhost:31010
token_type=jwt

We have now successfully set up and configured an External Token Provider for the Dremio SQL/JDBC endpoint.

Common Challenges

Additional Resources

https://docs.dremio.com/24.3.x/sonar/security/authentication/external-token

https://support.dremio.com/hc/en-us/articles/12776377653147-Configuring-KeyCloak-as-an-SSO