fix(core): fix spurious "cannot mmap" errors

[bluestreak01]

found by fuzz tests

This test after 4-6 mins running on Windows

    @Test
    public void testStressWalPurgeJob() throws Exception {
        // Here we generate many WAL segments and run WalPurgeJob frequently.
        // The goal is to make sure WalPurgeJob doesn't delete WAL-E files used by MatViewRefreshJob.
        setProperty(PropertyKey.CAIRO_WAL_SEGMENT_ROLLOVER_ROW_COUNT, 10);
        setProperty(PropertyKey.CAIRO_WAL_PURGE_INTERVAL, 10);
        assertMemoryLeak(() -> {
            Rnd rnd = fuzzer.generateRandom(LOG, 624968545800L, 1756738649271L);
            setFuzzParams(rnd, 0, 0);
            setFuzzProperties(rnd);
            runMvFuzz(rnd, getTestName(), 4);
        });
    }

---

Unrelated test failure:

2025-09-04T21:23:48.284128Z I i.q.t.TestListener >>>> io.questdb.test.cutlass.line.tcp.LineTcpReceiverTest.testGoodAuthenticated[WITH_WAL]
>>>>= io.questdb.test.cutlass.line.tcp.LineTcpReceiverTest.testGoodAuthenticated[WITH_WAL]
2025-09-04T21:23:48.341866Z I i.q.t.AbstractTest Starting test LineTcpReceiverTest#testGoodAuthenticated[WITH_WAL]
2025-09-04T21:23:48.352689Z I i.q.c.TableNameRegistryStore reloading tables file [path=C:\Users\VSSADM~1\AppData\Local\Temp\junit15334152461061910086\dbRoot\tables.d.0, threadId=4584]
2025-09-04T21:23:48.356254Z A IODispatcher listening on 0.0.0.0:9002 [fd=35564477030591 backlog=64]
2025-09-04T21:23:48.365008Z I i.q.c.t.t.InputFormatConfiguration loading input format config [resource=/text_loader.json]
2025-09-04T21:23:48.367900Z I i.q.t.c.l.t.LineTcpReceiverTest worker pool started [pool=testing]
2025-09-04T21:23:48.379665Z I IODispatcher connected [ip=127.0.0.1, fd=25703232118978, connCount=2]
2025-09-04T21:23:48.379954Z E i.q.c.l.t.LineTcpConnectionContext [25703232118978] could not parse measurement, NO_FIELDS at 9, line (may be mangled due to partial parsing): 'testUser1'
2025-09-04T21:23:48.412980Z I i.q.t.c.l.t.LineTcpReceiverTest os scheduled worker started [name=testing_0]

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The PR removes whole-file mapping helpers across memory classes, switches callers to explicit of(...) mapping with FilesFacade/LPSZ, defers index key-memory resizing to specific extension points, adjusts symbol map reader to map from an offset, and updates tests accordingly (including removing a too-small-file test and updating MV fuzz helpers).

Changes

Cohort / File(s)	Summary of Changes
Index key-memory mapping and resizing core/src/main/java/io/questdb/cairo/AbstractIndexReader.java, core/src/test/java/io/questdb/test/cairo/BitmapIndexTest.java	Replace wholeFile(...) with explicit of(...) using FilesFacade/LPSZ. Remove pre-open size check and eager resize. Add keyMem.extend at read/compute points and in updateKeyCount(). Tests updated to map via of(...), add reader.updateKeyCount() in one test, and remove file-too-small test.
Symbol values mapping core/src/main/java/io/questdb/cairo/SymbolMapReaderImpl.java	Switch from whole-file map plus extend to of(...) mapping starting at offsetMem.getLong(maxOffset) with explicit options.
Memory mapping API cleanup core/src/main/java/io/questdb/cairo/vm/MemoryCMRImpl.java, core/src/main/java/io/questdb/cairo/vm/MemoryPMARImpl.java, core/src/main/java/io/questdb/cairo/vm/NullMemoryCMR.java, core/src/main/java/io/questdb/cairo/vm/api/MemoryM.java	Remove wholeFile(...) convenience methods. Update MemoryCMRImpl.of(...) to treat size as final, compute newSize from file length when negative, and use newSize for mapping and assertions.
MV fuzz test adjustments core/src/test/java/io/questdb/test/cairo/mv/MatViewFuzzTest.java	Replace period-based MV helpers with timer-based ones. Add uniform MV existence check. Update fuzz params to new 3-arg signature (truncateProb=0). Remove period MV helpers.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Client
  participant IndexReader
  participant FilesFacade as FF
  participant KeyMem as MemoryM

  Client->>IndexReader: open(path)
  IndexReader->>FF: resolve LPSZ path, getMapPageSize()
  IndexReader->>KeyMem: of(FF, name, mapPageSize, start=0 or derived, tag, opts, madvise)
  IndexReader->>IndexReader: readIndexMetadataAtomically()
  Note right of IndexReader: After metadata read<br/>extend keyMem to getKeyEntryOffset(keyCount)
  alt nulls present
    IndexReader->>KeyMem: extend(getKeyEntryOffset(keyCountIncludingNulls))
  end
  Client->>IndexReader: updateKeyCount()
  IndexReader->>KeyMem: extend(getKeyEntryOffset(newKeyCount))

Loading

sequenceDiagram
  autonumber
  participant Client
  participant SymbolMapReader
  participant FilesFacade as FF
  participant OffMem as offsetMem
  participant CharMem as MemoryM

  Client->>SymbolMapReader: open(valuesPath, offsetsPath)
  SymbolMapReader->>OffMem: getLong(maxOffset)
  SymbolMapReader->>FF: getMapPageSize()
  SymbolMapReader->>CharMem: of(FF, valuesPath, mapPageSize, start=maxOffset, tag, opts, madvise)
  SymbolMapReader-->>Client: ready

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Suggested labels

Bug, Core

✨ Finishing Touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch vi_fix_index_reader_reload

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[bluestreak01]

@CodeRabbit review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

core/src/main/java/io/questdb/cairo/vm/MemoryCMRImpl.java (1)

47-49: Constructor calls outdated of(...) overload — compile error

of(...) now takes 7 params; the constructor passes 6. Add madviseOpts (e.g., -1).

-        of(ff, name, 0, size, memoryTag, 0);
+        of(ff, name, 0, size, memoryTag, 0, -1);

🧹 Nitpick comments (1)

core/src/main/java/io/questdb/cairo/vm/MemoryCMRImpl.java (1)

110-131: Make the double ff.length() check explicit; opt/extend params currently unused

The assertion intentionally calls ff.length(fd) twice. Encode that intent for readability and determinism in debug builds. Also, opts/extendSegmentSize are unused here; fine if by design, but consider a clarifying comment.

-            assert !VM_PARANOIA_MODE || newSize <= ff.length(fd) || newSize <= ff.length(fd); // Some tests simulate ff.length() to be 0 once.
+            // Intentionally call ff.length(fd) twice: some tests simulate a transient 0 once.
+            final long len1 = ff.length(fd);
+            final long len2 = ff.length(fd);
+            assert !VM_PARANOIA_MODE || newSize <= len1 || newSize <= len2;

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between e89d2fc and 4f2ecba.

📒 Files selected for processing (8)

• core/src/main/java/io/questdb/cairo/AbstractIndexReader.java (4 hunks)
• core/src/main/java/io/questdb/cairo/SymbolMapReaderImpl.java (1 hunks)
• core/src/main/java/io/questdb/cairo/vm/MemoryCMRImpl.java (1 hunks)
• core/src/main/java/io/questdb/cairo/vm/MemoryPMARImpl.java (0 hunks)
• core/src/main/java/io/questdb/cairo/vm/NullMemoryCMR.java (0 hunks)
• core/src/main/java/io/questdb/cairo/vm/api/MemoryM.java (0 hunks)
• core/src/test/java/io/questdb/test/cairo/BitmapIndexTest.java (17 hunks)
• core/src/test/java/io/questdb/test/cairo/mv/MatViewFuzzTest.java (10 hunks)

💤 Files with no reviewable changes (3)

• core/src/main/java/io/questdb/cairo/vm/MemoryPMARImpl.java
• core/src/main/java/io/questdb/cairo/vm/NullMemoryCMR.java
• core/src/main/java/io/questdb/cairo/vm/api/MemoryM.java

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)

• GitHub Check: New pull request (Hosted Running tests on windows-other)
• GitHub Check: New pull request (Coverage Report Coverage Report)
• GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-x64-zfs)
• GitHub Check: New pull request (SelfHosted Other tests on linux-x64-zfs)
• GitHub Check: New pull request (SelfHosted Other tests on linux-arm64)
• GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests on linux-arm64)
• GitHub Check: New pull request (SelfHosted Other tests Start X64Zfs Agent)
• GitHub Check: New pull request (Hosted Running tests on windows-other)
• GitHub Check: New pull request (Hosted Running tests on windows-pgwire)
• GitHub Check: New pull request (Hosted Running tests on windows-cairo)
• GitHub Check: New pull request (Hosted Running tests on windows-fuzz2)
• GitHub Check: New pull request (Hosted Running tests on windows-fuzz1)
• GitHub Check: New pull request (Hosted Running tests on windows-griffin)
• GitHub Check: New pull request (Hosted Running tests with cover on linux-other)
• GitHub Check: New pull request (Hosted Running tests on mac-other)
• GitHub Check: New pull request (Hosted Running tests with cover on linux-pgwire)
• GitHub Check: New pull request (Hosted Running tests on mac-pgwire)
• GitHub Check: New pull request (Hosted Running tests with cover on linux-cairo)
• GitHub Check: New pull request (Hosted Running tests on mac-cairo-fuzz)
• GitHub Check: New pull request (Hosted Running tests on mac-cairo)
• GitHub Check: New pull request (Hosted Running tests with cover on linux-fuzz)
• GitHub Check: New pull request (SelfHosted Other tests Start ARM Agent)
• GitHub Check: New pull request (Hosted Running tests on mac-griffin)
• GitHub Check: New pull request (Hosted Running tests with cover on linux-griffin)
• GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start X64Zfs Agent)
• GitHub Check: New pull request (SelfHosted Griffin And Fuzz tests Start ARM Agent)
• GitHub Check: New pull request (Rust Test and Lint on linux-jdk17)
• GitHub Check: New pull request (Check Changes Check changes)

🔇 Additional comments (12)

core/src/test/java/io/questdb/test/cairo/mv/MatViewFuzzTest.java (5)

73-73: LGTM! Consistent update to new fuzz parameter API.

The change from 2-argument to 3-argument setFuzzParams calls throughout the test file is consistent with the new API that includes truncateProb parameter.

---

189-189: Consistent initialization of truncateProb parameter to 0.

All test methods now consistently pass 0 as the truncateProb parameter in setFuzzParams calls, maintaining backward compatibility while adapting to the new API.

Also applies to: 214-214, 228-228, 398-398, 412-412, 427-427, 441-441, 953-953

---

626-633: Good addition of materialized view existence check.

Adding a preliminary check to verify that the materialized view exists before validating its status improves error reporting. This will help diagnose issues where views fail to be created entirely, separate from validation failures.

---

179-179: Truncate probability enabled in invalidation test.

Setting truncateProb to 0.5 in the invalidation test is appropriate as truncation should lead to materialized view invalidation, which is what this test is validating.

---

819-821: Well-structured API refactoring with backward compatibility.

The refactoring preserves the existing 2-argument helper by making it call the full 4-argument version with default values. This maintains backward compatibility while extending the API.

core/src/main/java/io/questdb/cairo/SymbolMapReaderImpl.java (1)

191-199: Critical fix: Symbol map memory now mapped from correct offset.

The change from charMem.wholeFile() followed by extendAndMap() to directly mapping from the offset stored at maxOffset is a key fix. This ensures the symbol values memory is correctly positioned, which would explain Windows-specific failures where memory mapping behavior may differ.

core/src/test/java/io/questdb/test/cairo/BitmapIndexTest.java (3)

65-65: LGTM! Import added for LPSZ usage.

The addition of the LPSZ import is necessary for the updated memory mapping approach using path.$().

Also applies to: 245-247

---

245-247: Improved memory mapping approach with explicit parameters.

The change from mem.wholeFile() to mem.of(...) with explicit FilesFacade, LPSZ path, and memory parameters aligns with the broader refactoring to use explicit region-based mappings instead of whole-file mappings.

---

689-690: Good fix: Update key count before backward scan.

Adding reader.updateKeyCount() before latestScanBackward ensures the reader has the latest key information. This is important when the index has been modified after the reader was created, preventing potential issues with stale metadata.

core/src/main/java/io/questdb/cairo/AbstractIndexReader.java (3)

206-207: Extending key memory on growth is correct

Deferring keyMem.extend() until keyCount increases avoids premature large mappings and matches the writer’s growth.

---

211-213: Comment clarity improvement

The triple-check explanation reads clearer now. No functional concerns.

---

235-235: Eager resize after header read is appropriate

Extending to the computed entry offset here reduces reload races.

[puzpuzpuz]

The PR title mentions mat views while the issue is not related with mat views, so I'd change the title - any table reader could fail on windows, not only readers open by mat view refresh.

[glasstiger]

[PR Coverage check]

😍 pass : 18 / 18 (100.00%)

file detail

	path	covered line	new line	coverage
🔵	io/questdb/cairo/SymbolMapReaderImpl.java	4	4	100.00%
🔵	io/questdb/cairo/TxReader.java	3	3	100.00%
🔵	io/questdb/cairo/vm/MemoryCMRImpl.java	4	4	100.00%
🔵	io/questdb/cairo/AbstractIndexReader.java	7	7	100.00%