Sparse support

[xori]

Utilizing the fswin package, upon opening a file, set the sparse flag in Windows, this operation takes 0.1ms on my computer so I believe it's safe to do on every file open.

I also included some tests that only run in Windows. I might have let it get out of hand with all the callbacks, but the callback style of this project triggered my nostalgia for a time before promises. Feel free to not include them, or let me know if you'd prefer me to remake them (or anything else) using modern technology or a different methodology.

This closes #23 and will be a pivotal step towards closing hypercore#281

[xori]

Hilariously, travis is failing due to nodejs/node#25913 where for the longest time node's fs.stat on windows returned blocks: undefined. Let me know if you'd like for me to handle the undefined or just scrap this test.

[xori]

Ignore my bellyaching, I was being an idiot. Tests pass, if you want any changes feel free to request them, or modify them to your heart's content.

[mafintosh]

Nice work, will review after I get my win laptop started 👨‍💻

[xori]

btw, if you want me to run any benchmarks or specific downstream tests with hypercore let me know.

[xori]

@mafintosh is it possible to merge this, or give direction if this isn't the solution you're looking for? I'm hoping to use this module with larger video files and this sparse flag becomes very useful when the filesize is larger than a GB.

[mafintosh]

I’ll try to land this, this weekend. Only concern is adding a big native dep, but I’ll see if we can extract what we need into a small napi module first

[RangerMauve]

It'd be awesome to have this. 💜

[mafintosh]

Yep getting close to being done factoring this into our existing native dep

[xori]

As a learning experience I made https://github.com/xori/set-sparse. I've never published a native module package before, so you won't hurt my feelings if you don't use it, but I thought I should share this with you.

Library size is 97.6k vs 489k from fswin, uses NAPI 3 so works for node >=8.11.2 and should work with all the stable electron versions. I chose to bundle in the compiled library because personally I think install speed compared to the extra time of an install script, for example, prebuild-install to save ~90kb is worth it.

[mafintosh]

@xori that's awesome! mind if integrate this into a small single one that has the native stuff we need (locking and win sparseness)?

In general we always use prebuildify for our native deps for the install speed bump plus the fallbacks.

[mafintosh]

@xori I included your code here https://github.com/mafintosh/fsctl so we can collect all the native utils we use in one tiny place. Note we only need it to work on fds so it's even simpler :)

[mafintosh]

Has windows, linux, osx and osx M1 prebuilts bundled. Need to get my rasperry running so I can build those as well.

Would anyone be interested in adding support for a sparse function option to the random-access-file module to allow us to pass this down? Similar to how we pass the lock function down.

[mafintosh]

(I added it in master)

[mafintosh]

Used in hypercore 9.9.0 if you use the default file storage