Summary
A reflection's expiration time is dependent on the source settings, which means that the reflections will get removed according to the expiration set there.
Relevant Versions
Dremio 24, 25, 26
Details
As a test, you can choose to set your source to never refresh your reflections and set the expiry date/time from the default 3 hours to 1 hour:
Now create a raw reflection on your VDS.
You will see entries in sys.reflections and sys.materializations and check creation date/time and expiry date/time by running the 2 queries below (beware of the TZ, which should be UTC):
select reflection_id, created_at, available_until from sys.reflections
select reflection_id, created, expires from sys.materializations
If you set you paths.accelerator = "dremioS3:/path_here/dremio/accelerator" in dremio.conf, you can check in S3 that a directory is created with a name corresponding to the reflection _id.
No wait until the expiration time has elapsed and check your s3 bucket. You will find that the reflections have been removed from S3.
server.log, audit.json and queries.json are also good sources of information by grepping for the reflection_id.
For example, you would see something like below in the server.log, once the expiration time has elapsed.:
2025-05-09 14:42:26,549 [reflection-manager] INFO c.d.s.reflection.ReflectionManager - Deleted
incremental materialization materialization
5ffdd4cd-0666-4033-ae32-8493b0ccf00e/fa678398-00c0-4d0e-8fbd-c6d8c0d23f2d without DROP TABLE job.
Extra information:
Also, if you turn off the reflection and then check sys.materializations, a change of state from DONE ( active) to DEPRECATED (disabled ) occurs.
The materialization will only be removed once expiration + reflection.deletion.grace_seconds (default being 14400 secs, ie 4 hours ) have elapsed.
This means that by default, deprecated reflections are removed 4 hours after they were marked as deprecated, which would allow any running query using the reflection to complete. A materialization is deprecated once a new materialization is available. Any cached plans referring to the old materialization are deprecated/invalidated so that they are not used again
Note that the reflection manager only drops 10 reflections at a time and refreshes every 30 seconds so expect a slight delay in the reflection removal.
REFRESH REFLECTION '5ffdd4cd-0666-4033-ae32-8493b0ccf00e' AS 'ee60e461-0aa8-4a8e-9cb8-64e565e3ee44'Be aware that the reflection drop take the following form in the jobs page: DROP TABLE IF EXISTS "__accelerator".
Additional Resources
https://www.dremio.com/wp-content/uploads/2024/01/Evaluating-Reflection-Usage.pdf