Small docs improvements #231
Conversation
docs/quickstart.md
Outdated
| @@ -23,6 +23,8 @@ Base. | |||
| -w /input/scientists.wordsfile.tsv \ | |||
| -d /input/scientists.docsfile.tsv | |||
| qlever@xyz:/app$ exit | |||
|
|
|||
| **Note for Mac users.** Having errors building an index follow [troubleshooting](./troubleshooting.md#note-for-mac-users). | |||
There was a problem hiding this comment.
This doesn't sound grammatically correct and I think it's a bit confusing calling "Docker on macOS" "Mac users". Maybe:
**Note:** If you experience an error during index building check the [troubleshooting](./troubleshooting.md) document for known issues such as problems with Docker on Mac.
There was a problem hiding this comment.
The "Mac users" should be fine. The problem is not in the docker itself but in APFS which is all Macs feature, so this should be also a problem running on native Mac env.
I can recheck this, but it can take some time run everything on native Mac. Do you think this is useful to try to run it on Mac?
There was a problem hiding this comment.
I don't think it makes sense for us to invest too much time in porting to macOS. I also think that Docker for Mac is the only variant that's really relevant here as it should work in principle. Arguing whether it would also be a problem when running directly on macOS is kind of moot since that's really not supported. But yeah I did mention file systems as one could indeed get the same problem on Linux when using e.g. a NTFS disk shared between Windows and Linux. So lets make this about the file systems and not about using Mac.
docs/troubleshooting.md
Outdated
| QLever preallocates a large (1TB) virtual file on disk for index. | ||
| This is done for performance reasons and | ||
| does not really allocate this amount on disk for standard Linux file systems (EXT3, etc). | ||
| However, this can cause an error on Mac APFS file system: |
There was a problem hiding this comment.
I don't think "virtual file" is the right word here, also EXT3 has been outdated for a while so we should say EXT4.
Maybe:
## Note when using Docker on Mac or Windows
When building an index, QLever will create scratch space for on-disc sorting. This space is allocated as a large 1 TB sparse file. On standard Linux file systems such as Ext4 sparse files are optimized and this will only take as much disk space as is actually used.
With some systems such as Docker on macOS or when using unsupported file systems such as NTFS or APFS, this may lead to problems as these do not properly support sparse files.
One possible error may be the following:
While macOS including Docker on Mac is not supported there are some workarounds.
You can manually change the constant `static const size_t STXXL_DISK_SIZE_INDEX_BUILDER` in [file](../src/global/Constants.h) or you can try using a named volume instead of a path on the host.
|
Corrected) |
Refs #225
Documented the issue with allocation of large file on APFS.