Benchmark: compacting a big dataset collection, and querying DB at the same time. All tests done after dropping VM cache. All tests repeated 3 times.

1. Performance when not compacting (baseline, best case performnance):

• 0:39
• 0:40
• 0:40

2. Performance when compacting (master):

• 1:03
• 1:07
• 1:11

3. Performance when compacting (after Lower iopriority when indexing to IDLE #213)

• 1:17
• 1:13
• 1:09

Yeah, on average the database got slower. But I realised that's because IO priority is per process, not per thread. And ursadb is a single (multi-threaded) process. So I can't actually do what I hoped to do.

But that's not all - I've tried to work around this by running a second ursadb process ("slow" process for compacting), and the results are

4. Performance when compacting (after Lower iopriority when indexing to IDLE #213, with separate process):

• 1:11
• 1:24
• 1:16

And this is... Even slower? This is surprising to me, I think this time all the priorities were installed as I wanted them to be.

But just to be sure I've ran a second DB by just manually changing the priority with ionice, and:

5. Performance when compacting (after Lower iopriority when indexing to IDLE #213, with separate process, one started by ionice -c 3):

• 1:10
• 1:10
• 1:15

I have to say this is underwhelming.

Also actually I also suspect that a big part of the slowdown comes from the fact that OS' disk cache is filled by useless (never again used) data. Maybe I should experiment with MADV_DONTNEED instead?

Anyway, looks like this approach may be more challenging than I suspected. Looks like I need to ponder on this topic a bit more 🤔

Initial tests suggest that adding fadvise may not help either:

• 1:03
• 1:16
• 1:15