You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
In Dora the file metadata is cached.
Imagine the below sequence:
We have file /a/b/c in Dora and by alluxio fs ls /a/b/c, the file metadata is cached in a Dora worker.
Then we recursively delete alluxio fs rm -R /a/. This will talk to a worker, and that worker will recursively remove /a/ from the UFS too.
Then we can still see alluxio fs ls /a/b/c from the cache, until that expires. At this stage, /a/ does not exist in UFS. But some of its children may still exist in cache.
Even if when we delete rm -R /a/, we clear all metadata about /a/* on that worker, that purging does NOT happen to all other workers. So if /a/b/c/ hashes to another worker (not the same one where we delete /a/ and all /a/* metadata), that /a/b/c cache may still exist.
Once that cache expires, or if we ls with a refresh option, ls will reflect the real state and complain the file does not exist. This is because if there is no cache, Dora worker will contact the UFS and realize the file does not exist.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Urgency
Explain why the feature is important
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
### What changes are proposed in this pull request?
The delete/create options are currently not passed to the UFS. By passing them to the UFS correctly, we should be able to achieve recursive create/delete.
However, note that under Dora structure, when we delete `/a/`, we can't also remove all `/a/*` cache from all workers immediately. #17741
pr-link: #17724
change-id: cid-fc5550beec4c9425505f624576b7580a9b171996
### What changes are proposed in this pull request?
The delete/create options are currently not passed to the UFS. By passing them to the UFS correctly, we should be able to achieve recursive create/delete.
However, note that under Dora structure, when we delete `/a/`, we can't also remove all `/a/*` cache from all workers immediately. Alluxio#17741
pr-link: Alluxio#17724
change-id: cid-fc5550beec4c9425505f624576b7580a9b171996
Is your feature request related to a problem? Please describe.
In Dora the file metadata is cached.
Imagine the below sequence:
/a/b/c
in Dora and byalluxio fs ls /a/b/c
, the file metadata is cached in a Dora worker.alluxio fs rm -R /a/
. This will talk to a worker, and that worker will recursively remove/a/
from the UFS too.alluxio fs ls /a/b/c
from the cache, until that expires. At this stage,/a/
does not exist in UFS. But some of its children may still exist in cache.Even if when we delete
rm -R /a/
, we clear all metadata about/a/*
on that worker, that purging does NOT happen to all other workers. So if/a/b/c/
hashes to another worker (not the same one where we delete/a/
and all/a/*
metadata), that/a/b/c
cache may still exist.Once that cache expires, or if we ls with a refresh option,
ls
will reflect the real state and complain the file does not exist. This is because if there is no cache, Dora worker will contact the UFS and realize the file does not exist.Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Urgency
Explain why the feature is important
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: