Skip to content

Add Pagefind incremental indexing#2890

Open
MoshiMoshiMochi wants to merge 9 commits intoMarkBind:masterfrom
MoshiMoshiMochi:dev/pagefind-incremental-indexing
Open

Add Pagefind incremental indexing#2890
MoshiMoshiMochi wants to merge 9 commits intoMarkBind:masterfrom
MoshiMoshiMochi:dev/pagefind-incremental-indexing

Conversation

@MoshiMoshiMochi
Copy link
Copy Markdown
Contributor

@MoshiMoshiMochi MoshiMoshiMochi commented Apr 13, 2026

What is the purpose of this pull request?

  • Documentation update
  • Bug fix
  • Feature addition or enhancement
  • Code maintenance
  • DevOps
  • Improve developer experience
  • Others, please explain:

Overview of changes:
closes #2873
Implemented incremental Pagefind search index

Anything you'd like to highlight/discuss:
The incremental Pagefind search index works in the following manner.

  1. If a user updates a page -> Pagefind will re-index that specific page.
  2. If a user creates a new page -> Pagefind will index that specific new page.
  3. If a user deletes a page -> Pagefind will re-index all pages.

For deleting I couldn't find a way to remove just 1 specific index file from in-memory indexes. Hence, I decided that the remove handler will use indexSiteWithPagefind, meaning that it will reindex the entire site. There might be a better solution out there. But since removing already calls rebuildSourceFiles which takes significantly longer than it takes to reindex the site, I don't think its a big deal. But hey just something to note if possible to improve ig.

Technically this some what tackles #2882 but it does so indirectly as we no longer call indexSiteWithPagefind when serving one-page, instead we call updatePagefindIndex which incrementally indexes pages.

Testing instructions:
Full build: markbind serve -d

  1. Serve any markbind site
  2. Update any page and save the changes -> Pagefind should only index that one page -> Check to see if the changes are within the search
  3. Add any page -> Pagefind should only index that newly added page -> Check to see if the newly create page is searchable by Pagefind
  4. Delete any page -> Pagefind should re-index the entire site -> Check to see if the removed page is no longer searchable

One-Page: markbind serve -o

  1. Serve any markbind site with just one-page -> Pagefind should index that specific page if it is searchable
  2. Navigate through the other pages -> Pagefind should index those pages incrementally

Proposed commit message: (wrap lines at 72 characters)
Add Pagefind incremental indexing


Checklist: ☑️

  • Updated the documentation for feature additions and enhancements
  • Added tests for bug fixes or features
  • Linked all related issues
  • No unrelated changes

Reviewer checklist:

Indicate the SEMVER impact of the PR:

  • Major (when you make incompatible API changes)
  • Minor (when you add functionality in a backward compatible manner)
  • Patch (when you make backward compatible bug fixes)

At the end of the review, please label the PR with the appropriate label: r.Major, r.Minor, r.Patch.

Breaking change release note preparation (if applicable):

  • To be included in the release note for any feature that is made obsolete/breaking

Give a brief explanation note about:

  • what was the old feature that was made obsolete
  • any replacement feature (if any), and
  • how the author should modify his website to migrate from the old feature to the replacement feature (if possible).

It takes in an array of pages that needs to be updated. Filters if the
page is need searchable, then add a new HTML file to update the
in-memory index. Finally, we retrieve all the in-memory indexes, wipe
the old files and write newly updated files to disk.
add & change handlers use updatePagefindIndex as there is only a need to
either add/update 1 file in-memory.

However, as there is no way of to remove just 1 specific index file from
in-memory indexes, remove handler will use indexSiteWithPagefind,
meaning that it will reindex the entire site. There might be a better
solution out there. But since removing already calls rebuildSourceFiles
which takes significantly longer that it takes to reindex the site. I
don't think its a big deal.
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 13, 2026

Codecov Report

❌ Patch coverage is 84.37500% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.75%. Comparing base (b3b90c8) to head (555d1c3).

Files with missing lines Patch % Lines
packages/core/src/Site/SiteGenerationManager.ts 82.45% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2890      +/-   ##
==========================================
+ Coverage   71.35%   71.75%   +0.39%     
==========================================
  Files         133      133              
  Lines        7229     7285      +56     
  Branches     1611     1720     +109     
==========================================
+ Hits         5158     5227      +69     
+ Misses       2065     2052      -13     
  Partials        6        6              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Now serving one page will directly call updatePagefindIndex with the
respective page to build instead of calling indexSiteWithPagefind
which will attempt to index all other pages too, even if they DNE.

Still need to update test cases... arghhhhhhh lol
Removed warning about how pagefind only works on full site build as that
is no longer the case after these changes
@MoshiMoshiMochi MoshiMoshiMochi marked this pull request as ready for review April 13, 2026 17:20
@MoshiMoshiMochi MoshiMoshiMochi requested a review from a team April 13, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement incremental Pagefind indexing for markbind serve

1 participant