-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Files uploaded to a CSI volume are not visible in Filer #127
Comments
added a fix. Need to wait for a new image. |
I installed 3.54 but my files are still not visible neither in the Filer web UI nor via fs.meta.save. |
I deployed chrislusf/seaweedfs-csi-driver:v1.1.7, then restarted all the jobs that are associated with the seaweedfs volumes and they appeared in the Filer according to the path value:
So now I have /buckets/app_content (old) and /buckets/volume_app_content (new). Should I delete /buckets/app_content manually ? |
Not clear how the nomad template works here. |
Full volume template (volume_app_content.hcl):
It was passed to nomad with the command:
So, after the 1.1.7 deployment, I didn't make any changes in nomad. Just restarted jobs via UI. And volumes became visible in Filer, in weed mount and in fs.meta.save But now I have a strange error with fs.meta.save (after it does some work): error: rpc error: code = Unknown desc = list /buckets/volume_app_content/shop/products/9993/photos : dial tcp 192.168.3.22:33306: connect: cannot assign requested address 33306 - is a maxscale port (MariaDB proxy). There are no other errors in syslog, maxscale log or mariadb log.
|
It seems like mysql driver has unlimited connections by default.
20 seconds after fs.meta.save:
When I updated Filer config to
it started working without errors. |
So finally I changed storage from mysql to redis3 (keydb active-active replication) and imported the meta. Performance measurements: MariaDB:
KeyDB 1-st run after meta load:
KeyDB 2-nd run after meta load:
The redis3 speed is almost the same as MariaDB. Is that OK? BTW, it would be great if Thank you for the excellent product! |
I decided to exclude possible cache influence and tested again with MariaDB.
2-nd run:
So it seems like redis3/KeyDB actually faster than MariaDB (1m53.215s <= 2m44.354s). |
It seems like redis have a problem:
We loading it to redis (keydb):
then we immediately saving it:
it saves less files and directories that were loaded. |
redis2 doesn't have such a problem:
redis2: fs.meta.load was finished in 7 minutes second test with redis3 didn't show that problem:
MySQL also was tested on a clean database - fs.meta.load completed in 1 hour. So redis2 is the fastest storage. |
I can see only volume names but not their contents. Same issue is with weed mount or weed shell fs.meta.save.
I use mysql Filer backend and meta data is available for all uploaded files:
The text was updated successfully, but these errors were encountered: