Skip to content

How to use MMAP to optimize data reading

Andrei Grazhdankov edited this page May 24, 2021 · 9 revisions

STDIN is a file in RAM. That's why you can use STDIN_FILENO to get access to the file. Now you can create a mapping to this file in the virtual address space of the calling process using the mmap function.

C++:

    #include <unistd.h>
    #include <sys/mman.h>
    ...
    off_t fsize = lseek(0, 0, SEEK_END);
    char* buffer = (char*)mmap(0, fsize, PROT_READ, MAP_PRIVATE | MAP_POPULATE, 0, 0);

Go:

    f := os.NewFile(0, "stdin")
    fsize, _ := f.Seek(0, os.SEEK_END)
    buffer, _ := syscall.Mmap(0, 0, int(fsize), syscall.PROT_READ, syscall.MAP_PRIVATE | syscall.MAP_POPULATE)

Rust:

https://gist.github.com/stremin/89fb80a03808c8e1710d21512341adcd

Also the Huge Pages feature is available.

Clone this wiki locally