Add totalSize and totalCount values to VFS stats #66
Conversation
|
Just I cant not fix the unit-test of |
mahsashadi
changed the title
| filename, | ||
| stat | ||
| }); | ||
| const createStat = async stat => { |
There was a problem hiding this comment.
This method of figuring out the total size of a directory seems very inefficient, especially when there's a lot of files.
Maybe it would be better looking into spawning a system process to do this in a native way ? 🤔
There was a problem hiding this comment.
As I searched, du can be used liked this:
const child = spawn('sh', ['-c', 'du -s ${realPath} | cut -f1']);But the problem is that du seems to have considerable delay, especially for large directories.
There was a problem hiding this comment.
especially for large directories.
But I would assume it's faster than doing it with node, right?
There was a problem hiding this comment.
Also I'm not 100% sure, but I assume that this wouldn't be cross-compatible with users hoping to run OS.js locally on Windows. IMHO native system processes shouldn't be included without a fallback for Windows
There was a problem hiding this comment.
There's a du npm package that handles Windows etc.
There was a problem hiding this comment.
In new commit du package is used for totalSize. @andersevenrud
There was a problem hiding this comment.
I am thinking about our own vfs system adapter, which is a cloud object storage.
Since totalSize is not provided in APIs, I have to iterate through files which as you said, gets lots of time. Is there any other solution?
There was a problem hiding this comment.
Sorry, I can't think of a good solution to that, especially if there's pagination involved :/
There was a problem hiding this comment.
Obviously I'm going to defer to @andersevenrud since he is the SME, but this looks better and simplified to me. Good work!
|
Hi again @andersevenrud |
|
@mahsashadi I've just been too busy to keep up with all of the PRs :( I've made a notification in my calendar for this weekend so I can test and approve this. |
|
Hi again after long time. |
There was a problem hiding this comment.
It's been a long time indeed @mahsashadi. If only I had more time to follow up on things 😅
This looks good to me, but there's two very minor things:
- Could you add a configuration option for this (and keep it on by default) ?
- Resolve the conflict so it can be merged
Due to questions mentioned here, and also solutions provided here and here, we might need totalCount and totalSize in stats of directories.