-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: Not all files/folders shown / Randomly missing files/folders #47
Comments
How strange... My account has 3481 files (as reported by the logs) and Do you notice anything special about the files which change from one run to the next? This script should show you the differences: #!/bin/bash
set -x
mountpoint="/home/ttl/gdrive"
session="your_session_name"
first_ls=$(mktemp)
second_ls=$(mktemp)
gcsf mount ${mountpoint} -s ${session} &
sleep 30
find ${mountpoint} | sort > ${first_ls}
fusermount -u ${mountpoint}
gcsf mount ${mountpoint} -s ${session} &
sleep 30
find ${mountpoint} | sort > ${second_ls}
fusermount -u ${mountpoint}
vimdiff ${first_ls} ${second_ls} |
hello and thank you for the quick reply! I cannot find any pattern, its random I would say - every time other files or whole folders are missing. Files/folder which were missing can appear on next mount, so there seems to be no "blacklist" or something. Can you give me a little howto maybe whats happening after the populate() function? I think there lies the problem and I really want to fix it or at least help you fixing it |
The
These are required because each file can be identified in multiple ways (by inode, by parent id + name, by drive_id, by node_id from the tree). Initially, When you
My first action would be to add |
i followed your suggestion and added the println, which shows the correct files and counts, in this case a folder with around ~100 files inside:
those are 107 files/subfolders, which is correct. anyway the filesystem does not show them all: ttl@spin8:/home/ttl# ls gdrive/safe/BOm75A7a2sIP3Bw4hXbwAHnbMAA76tgViQjOD_MYinWuaeUi339pt6KY_8jbz55hNQHf2sVPRGKCh8_88xqS4b7Vo3gYa1srN3HtAn70HpsnsksFakeH4w1Sh_w77UUv/ | sed 's/ /\n/'
returns 63 I also touched 1024 normal txt-files in a fresh folder, those are all shown everytime. maybe the cryptic file names are problematic? will try to investigate the LOOKUP procedure further now and let you know merry XMAS! |
I added a few more println's and found out that readdir() is doing everything correctly - each file and folder is reported and "reply.add" executed. Now I am unaware of where to debug further, can you please help? |
I investigated the rust fuse library and found that the readdir() has a buffer of 4096, then added a println if the reply.add() function returns true or false:
I found out that the files which are not shown, reply "true", which means the buffer is full - but sadly I found no way to increase this buffer - I even tried editing the reply.rs from the rust-fuse library without luck. Hope you can help me solve this :) Good night |
good news: found and fixed the bug! problem is here: Lines 186 to 187 in c8d8406
there is no check if the buffer is already full(the add function returns True in this case) - thats why all files which doesnt fit the buffer dont get visible to the kernel. I added a break condition which works just fine. Please implement it in the main code - and also please add a config option for this "identical file id" stuff, which is useless for most people and causes problems when the underlying FS relies on special files. See attached patch and have a nice day :) https://gist.github.com/platinum55/d361e90cbf06f8865f041aad4f2ee163#file-gistfile1-txt |
I have ~7200 files in my drive and noticed that with GCSF mount, there are less shown. When looking further I noticed, that every mount the file count changes - even if my gdrive didnt change:
INFO gcsf > Creating and populating file system...
DEBUG gcsf::gcsf::drive_facade > DriveFacade::new()
INFO gcsf::gcsf::drive_facade > Received page 1 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 2 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 3 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 4 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 5 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 6 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 7 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 8 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 9 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 10 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 11 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 12 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 13 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 14 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 15 containing 460 files
INFO gcsf::gcsf::drive_facade > Received page 16 containing 224 files
INFO gcsf::gcsf::drive_facade > Received page 1 containing 5 files
INFO gcsf > File sytem created.
INFO gcsf > Mounting to /home/ttl/gdrive
INFO fuse::session > Mounting /home/ttl/gdrive
The mounting procedure every times shows the correct file count, when summarizing the reported file counts per page- Directly after each mounting I checked file/folder count:
find /home/ttl/gdrive | wc -l
5827
find /home/ttl/gdrive | wc -l
6099
find /home/ttl/gdrive | wc -l
6225
find /home/ttl/gdrive | wc -l
6136
there dont seem to be a fixed limit or something, because the shown file-count changes on every mount.
I looked into the code to find out whats happening and just added a counter in the populate() function - which is reporting the correct file count. The problem must be deeper inside, where my experience is too low
please help :)
The text was updated successfully, but these errors were encountered: