Things to check when you see the error : java.lang.RuntimeException: java.io.IOException: Incomplete HDFS URI

Summary/Reported Issue

You may encounter a failure when running a query, displaying the error: java.lang.RuntimeException: java.io.IOException: Incomplete HDFS URI, no host: <path>

Relevant Versions

All

Troubleshooting Steps

Check  the server.log files  and check the query is failing due to an incorrect uri .

Cause

An example of an  error we may see:

Incomplete HDFS URI, no host: hdfs:/path/to/data/database/table/partition_key=2025-03-06/source_key=example_source/some_file_id

Why are we seeing this?

Your query is selecting from a Hive table with partitions:

SELECT count(*) AS total_records,
       as_of_dt,
       'EXAMPLE_LAYER' AS layer,
       'EXAMPLE_TYPE' AS layer1,
       'SOURCE_SYSTEM' AS SOR,
       'TABLE_DETAILS' AS table_details
FROM hive.database_name.table_name
WHERE as_of_dt = '2025-03-06'
  AND prod_data_src = 'example_source'
GROUP BY as_of_dt

When Dremio executes this query, it:

• Identifies the Hive table (hive.database_name.table_name).
• Applies the partition filters (as_of_dt = '2025-03-06' and prod_data_src = 'example_source').
• Retrieves the physical location of the partition from the Hive metastore.
• Attempts to access the data files in that location.

The problem occurs in step 4, where the partition location returned from the Hive metastore contains an improperly formatted HDFS URI. Instead of a full URI like hdfs://hostname:port/path/to/data/..., the returned path is missing the host, causing the "Incomplete HDFS URI, no host" error.

How to check and resolve:

We need to review how the Hive metastore is configured.

1. Connect to the Hive CLI (in your Hive environment):

   $ hive

   Check the table location:

   DESCRIBE FORMATTED database_name.table_name;

   In the output, look for the "Location:" field. If it starts with hdfs:/ (one slash) instead of hdfs://hostname, that's the issue.

2. If the base location is correct, check the partition location:

   SHOW PARTITIONS database_name.table_name;
   
   DESCRIBE FORMATTED database_name.table_name PARTITION (as_of_dt='2025-03-06', prod_data_src='example_source');

   Again, look for the "Location:" field. If it starts with hdfs:/path, that's the problem.

3. You can also use Beeline as an alternative (changing server/port/credentials as needed):

   beeline -u "jdbc:hive2://hive-server-host:10000" -n username -p password

   Then run the same commands as above.

4. If you have direct SQL access to the Hive metastore database, you can run:

   -- For table location
   SELECT TBL_NAME, DB_NAME, LOCATION
   FROM DBS d
   JOIN TBLS t ON d.DB_ID = t.DB_ID
   JOIN SDS s ON t.SD_ID = s.SD_ID
   WHERE d.NAME = 'database_name' AND t.TBL_NAME = 'table_name';
   
   -- For partition locations
   SELECT PART_NAME, LOCATION
   FROM PARTITIONS p
   JOIN SDS s ON p.SD_ID = s.SD_ID
   WHERE p.TBL_ID = (
       SELECT TBL_ID FROM TBLS WHERE TBL_NAME = 'table_name'
           AND DB_ID = (SELECT DB_ID FROM DBS WHERE NAME = 'database_name')
   );

   In all these cases, look for locations that start with hdfs:/ (single slash, missing host) instead of the correct hdfs://hostname:port/ (double slash with host).

How to fix:

Once the improper location is identified, coordinate with your Hive team to correct it. For example:

ALTER TABLE database_name.table_name
SET LOCATION 'hdfs://namenode-host:port/path/to/data/database/table';

Or for a specific partition:

ALTER TABLE database_name.table_name PARTITION (as_of_dt='2025-03-06', prod_data_src='example_source')
SET LOCATION 'hdfs://namenode-host:port/path/to/the/partition/';

Make sure any SQL changes are validated and executed by your Hive administrators.