Overview
The use of a colon in the pathname of an S3 bucket when using Iceberg can cause an error in earlier versions of Dremio.
Applies To
Dremio Versions earlier than 19.8.0, 20.0.4, 21.2.0, 22.0.0
Details
The user may see the following error in the raw profile of a job:
[Error Id: d4edd081-060e-41ea-8cb9-f4a2c6fba833 on 10.8.2.72:0]
(java.lang.IllegalArgumentException) java.net.URISyntaxException: Relative path in absolute URI: broadcast date=2018-10-01 00:00:00
com.dremio.io.file.Path.toURI():381
com.dremio.io.file.Path.of():91
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.isValidPath():344
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.iterateDirectory():218
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.next():166
com.dremio.sabot.op.scan.ScanOperator.outputData():324
com.dremio.sabot.driver.SmartOp$SmartProducer.outputData():521
com.dremio.sabot.driver.StraightPipe.pump():56
com.dremio.sabot.driver.Pipeline.doPump():108
com.dremio.sabot.driver.Pipeline.pumpOnce():98
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():345
com.dremio.sabot.exec.fragment.FragmentExecutor.run():294
com.dremio.sabot.exec.fragment.FragmentExecutor.access$1600():94
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():747
com.dremio.sabot.task.AsyncTaskWrapper.run():112
com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop():243
com.dremio.sabot.task.slicing.SlicingThread.run():171
Caused By (java.net.URISyntaxException) Relative path in absolute URI: broadcast date=2018-10-01 00:00:00
java.net.URI.checkPath():1823
java.net.URI.<init>():745
com.dremio.io.file.Path.toURI():379
com.dremio.io.file.Path.of():91
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.isValidPath():344
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.iterateDirectory():218
com.dremio.exec.store.metadatarefresh.dirlisting.DirListingRecordReader.next():166
Cause
A defect (described in internal Jira DX-43201) where Dremio fails to handle a colon : character in S3 pathnames.
Solution
Upgrade to Dremio version 19.8.0, 20.0.4, 21.2.0, or 22.0.0 onwards which include the fix for this issue.