Summary
This article discusses how to resolve the NullPointerException or Cannot invoke "java.lang.Long.longValue()" because the return value of "com.dremio.service.namespace.dataset.proto.DatasetConfig.getLastModified()" is null error when running INFORMATION_SCHEMA.SCHEMATA queries after upgrading dremio from v24.3.x to v25.2.x.
Reported Issue
When attempting to run SELECT * (SELECT * FROM INFORMATION_SCHEMA.SCHEMATA) or COUNT (*) (SELECT COUNT(*) FROM INFORMATION_SCHEMA.SCHEMATA) queries against the INFORMATION_SCHEMA.SCHEMATA view, these would fail in Dremio with the below error.
SELECT *:
VALIDATION ERROR: NullPointerException
COUNT(*)
Cannot invoke "java.lang.Long.longValue()" because the return value of "com.dremio.service.namespace.dataset.proto.DatasetConfig.getLastModified()" is null
Relevant Versions
This issue applies to Dremio versions 25.2.0 to 25.2.9.
Troubleshooting Steps
1. Navigate to the failing job profile as per the following documentation link steps: https://docs.dremio.com/current/sonar/monitoring/jobs/raw-profile/
2. After navigating to the "Raw Profile", click on the "Error" tab of the profile.
3. Confirm that the error is the same as mentioned in this article. As an example, the following error can be seen in the "Verbose" section of the "Error" tab of the job profile:
VALIDATION ERROR: NullPointerException
SQL Query select *
from INFORMATION_SCHEMA.SCHEMATA
ErrorOrigin: COORDINATOR
(java.lang.NullPointerException) null
com.dremio.exec.catalog.SourceMetadataManager.getDatasetMetadataState():494
com.dremio.exec.catalog.ManagedStoragePlugin.getDatasetMetadataState():1238
com.dremio.exec.catalog.DatasetManager.getTableFromNamespace():448
com.dremio.exec.catalog.DatasetManager.getExistingTableFromNamespace():580
com.dremio.exec.catalog.DatasetManager.lambda$bulkGetTablesFromPlugin$14():644
com.dremio.common.concurrent.bulk.BulkResponse$Response.lambda$transformAsync$2():312
com.dremio.common.concurrent.bulk.BulkResponse$Response.timedCall():380
com.dremio.common.concurrent.bulk.BulkResponse$Response.lambda$transformAsync$3():312
java.util.concurrent.CompletableFuture.uniComposeStage():1106
java.util.concurrent.CompletableFuture.thenCompose():2241
java.util.concurrent.CompletableFuture.thenCompose():143
com.dremio.common.concurrent.bulk.BulkResponse$Response.transformAsync():311
com.dremio.common.concurrent.bulk.BulkRequest.lambda$bulkTransformAndHandleRequestsAsync$11():279
java.lang.Iterable.forEach():75
com.dremio.common.concurrent.bulk.BulkRequest.lambda$bulkTransformAndHandleRequestsAsync$12():275
java.lang.Iterable.forEach():75
com.dremio.common.concurrent.bulk.BulkResponse.forEach():84
com.dremio.common.concurrent.bulk.BulkRequest.bulkTransformAndHandleRequestsAsync():272
com.dremio.exec.catalog.DatasetManager.bulkGetTablesFromPlugin():635
com.dremio.exec.catalog.DatasetManager.lambda$bulkGetTablesByCanonicalKey$6():333
com.dremio.common.concurrent.bulk.BulkRequest.bulkTransformAndHandleRequests():232
com.dremio.common.concurrent.bulk.BulkRequest.lambda$bulkPartitionAndHandleRequests$14():327
java.util.stream.ReferencePipeline$3$1.accept():195
java.util.HashMap$EntrySpliterator.forEachRemaining():1764
java.util.stream.AbstractPipeline.copyInto():484
java.util.stream.AbstractPipeline.wrapAndCopyInto():474
java.util.stream.ReduceOps$ReduceOp.evaluateSequential():913
java.util.stream.AbstractPipeline.evaluate():234
java.util.stream.ReferencePipeline.collect():578
com.dremio.common.concurrent.bulk.BulkRequest.bulkPartitionAndHandleRequests():330
com.dremio.exec.catalog.DatasetManager.bulkGetTablesByCanonicalKey():350
com.dremio.exec.catalog.DatasetManager.lambda$bulkGetTables$4():300
com.dremio.common.concurrent.bulk.BulkRequest.bulkTransformAndHandleRequests():179
com.dremio.exec.catalog.DatasetManager.bulkGetTables():299
com.dremio.exec.catalog.CatalogImpl.bulkGetTables():285
com.dremio.exec.catalog.CatalogImpl.bulkGetTablesForQuery():321
com.dremio.exec.catalog.EnterpriseCatalogImpl.bulkGetTablesForQuery():376
com.dremio.common.concurrent.bulk.BulkRequest.bulkTransformAndHandleRequests():232
com.dremio.common.concurrent.bulk.BulkRequest.lambda$bulkPartitionAndHandleRequests$14():327
java.util.stream.ReferencePipeline$3$1.accept():195
java.util.HashMap$EntrySpliterator.forEachRemaining():1764
java.util.stream.AbstractPipeline.copyInto():484
java.util.stream.AbstractPipeline.wrapAndCopyInto():474
java.util.stream.ReduceOps$ReduceOp.evaluateSequential():913
java.util.stream.AbstractPipeline.evaluate():234
java.util.stream.ReferencePipeline.collect():578
com.dremio.common.concurrent.bulk.BulkRequest.bulkPartitionAndHandleRequests():330
com.dremio.exec.catalog.SourceAccessChecker.bulkGetTableIfVisible():229
com.dremio.exec.catalog.SourceAccessChecker.bulkGetTablesForQuery():245
com.dremio.exec.catalog.CachingCatalog.lambda$getOrLoadTables$19():373
com.dremio.exec.catalog.CachingCatalog.handleException():410
com.dremio.exec.catalog.CachingCatalog.getOrLoadTables():373
com.dremio.exec.catalog.CachingCatalog.bulkGetTablesForQuery():269
com.dremio.exec.ops.TableMetadataPrefetcher.prefetch():160
com.dremio.exec.ops.TableMetadataPrefetcher.<init>():92
com.dremio.exec.ops.TableMetadataPrefetcher$Builder.build():307
com.dremio.exec.ops.IcebergAsyncMetadataRefreshHandler.bulkGetMetadata():291
com.dremio.exec.ops.IcebergAsyncMetadataRefreshHandler.prefetchTableMetadata():112
com.dremio.exec.planner.sql.handlers.EnterpriseNormalHandler.prePlan():58
com.dremio.exec.planner.sql.handlers.query.NormalHandler.getPlan():93
com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan():56
com.dremio.exec.work.foreman.AttemptManager.plan():754
com.dremio.exec.work.foreman.AttemptManager.lambda$run$4():589
com.dremio.service.commandpool.ReleasableBoundCommandPool.lambda$getWrappedCommand$1():156
com.dremio.service.commandpool.CommandWrapper.run():73
com.dremio.context.RequestContext.run():103
com.dremio.common.concurrent.ContextMigratingExecutorService.lambda$decorate$4():246
com.dremio.common.concurrent.ContextMigratingExecutorService$ComparableRunnable.run():222
java.util.concurrent.Executors$RunnableAdapter.call():515
java.util.concurrent.FutureTask.run():264
java.util.concurrent.ThreadPoolExecutor.runWorker():1128
java.util.concurrent.ThreadPoolExecutor$Worker.run():628
java.lang.Thread.run():829
4. Carry out the "Steps to Resolve" described below.
Cause
This was an issue (NPE) in correctly retrieving the dataset metadata for sources/spaces when querying the INFORMATION_SCHEMA.SCHEMATA view. Known issue reported internally against DX-103489.
Steps to Resolve
Upgrade to Dremio v25.2.10 or higher.
Additional Resources
Viewing a Raw Profile: https://docs.dremio.com/current/sonar/monitoring/jobs/raw-profile/
INFORMATION_SCHEMA.SCHEMATA View: https://docs.dremio.com/25.x/reference/sql/information-schema/schemata/
v25.2.10 Release Notes: https://docs.dremio.com/current/release-notes/version-250-release#improvements-and-issues-fixed