Pull out custom File from fast.json?

[baryluk]

I noticed you use mmap in fast.json.

It is a nice trick, but also doesn't need to be.

1. Refactor it so this mmap file implementation is in separate module, fast.file ? It uses alias m_json this; Json m_json;, but it can simply be a template parameter, so it can still leave be outside of json module, and json module would simply isntantiate it FastFile!Json for example. It looks to be static class/struct, so that is all. If you need access to some outer class variables, this can still be as template in separate module and used using mixing mixin FastFile!Json file;, with protocol defined and documented.

2. There is std.mmfile which is working nice, and has read-only mode too, so it should be used instead. If there are some flags (like madvise, etc) that makes a difference they could be upstramed to std.mmfile as options.

3. For small files it probably doesn't make sense to use mmap, as it can be expensive, not scale with number of cores / threads, and waste memory. Each mmap will probably mmap whole 4kB page, if left there for long. So for small files, it is better to just read explicitly into the properly-sized buffer, or even only do so for strings. A custom arena allocator can also be used. Yes, there is a function to parse from memory block, but then you put a lot of extra logic on the caller side, instead making it in a library.

4. What about reading a stream from network socket, Unix pipe, or from decompression library in chunks? It can't be easily mmaped, but is often an important application of parsing, and probably most common use case. Right now the only option is to buffer whole thing, and then parse, which in some synthetic scenarios can be about 2x more memory usage.

[mw66]

@baryluk maybe you want create a PR? Just-Do-It :-)