feat: add lazyLoad option to fetch the search index on demand

When enabled via params.modules.flexsearch.lazyLoad, the FlexSearch
index data is emitted as a separate per-language JSON file through a
new searchindex Hugo output format and fetched on the visitor's first
search interaction, instead of being bundled into the core script
loaded on every page. Cuts the core bundle by several megabytes on
content-heavy sites at the cost of a single fetch on first search.

The searchindex output format must be attached to home outputs at
the theme level — Hinode includes this wiring from the next release.
Sites with a strict Content Security Policy must allow connect-src
'self'; the module declares this in its csp block so mod-csp picks
it up automatically.

The eager and lazy paths share a single source of truth for index
documents (layouts/_partials/utilities/GetSearchDocs.html), so both
behave identically with respect to the existing summaryOnly,
frontmatter, filter, and canonifyURLs options.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: markdumay

CLAUDE.md

@@ -51,18 +51,26 @@ The module imports `github.com/nextapps-de/flexsearch` and mounts its bundle to
 - `layouts/_partials/assets/search-input.html`: Embedded search form HTML structure
 - `layouts/_shortcodes/ModalSearch.html`: Modal search dialog structure
 - `layouts/_partials/assets/search-meta.html`: Recursive helper to extract frontmatter content for indexing
+- `layouts/_partials/utilities/GetSearchDocs.html`: Single source of truth — builds the slice of index documents (consumed by both the eager JS and the lazy JSON output)
+- `layouts/index.searchindex.json`: Layout for the `searchindex` output format — emits the index documents as JSON on the home page (lazy mode)
+- `layouts/_partials/utilities/GetSearchIndex.html`: Returns the URL of the search-index output (lazy mode)
 - `assets/js/modules/flexsearch/flexsearch.index.js`: FlexSearch initialization and search logic
 
 **Search index configuration:**
-- The JavaScript file generates a FlexSearch document index at build time from Hugo's page content
+- The index documents are built by `GetSearchDocs.html` from Hugo's page content
 - Indexes three fields: `title` (forward tokenization), `description`, and `content` (full tokenization)
 - Supports optional frontmatter indexing via `flexsearch.frontmatter` parameter
 - Can index page summaries instead of full content via `flexsearch.summaryOnly` parameter
 - Can use absolute URLs via `flexsearch.canonifyURLs` parameter
 - Pages can be excluded with `searchExclude: true` in frontmatter
 - Supports optional `indexTitle` parameter to override title in search results
-- Each indexed page is emitted as a separate `index.add(...)` statement (a chained
-  `.add()` expression overflows the minifier's expression-nesting limit on large sites)
+- **Eager mode (default):** `flexsearch.index.js` emits one flat `index.add(...)`
+  statement per page into the core bundle (a chained `.add()` expression
+  overflows the minifier's expression-nesting limit on large sites)
+- **Lazy mode (`flexsearch.lazyLoad`):** the index is emitted as a separate
+  per-language JSON file via the `searchindex` output format and fetched on
+  the first search interaction (a normal page render, so page content can be
+  aggregated safely — unlike a detached resource template)
 
 **Search behavior:**
 - Shows up to 5 results across title, description, and content fields
@@ -85,6 +93,7 @@ Module configuration in `config.toml` under `params.modules.flexsearch`:
 - `frontmatter` (default: false): Include frontmatter content in search index
 - `filter` (default: "params"): Restrict frontmatter scanning to specific key
 - `summaryOnly` (default: false): Index page summaries instead of full content
+- `lazyLoad` (default: false): Emit the index as a separate JSON file fetched on first search interaction, instead of bundling it into every page
 - `localize` (default: true): Enable language-specific search
 
 Navigation configuration under `params.navigation.search`:

README.md

@@ -48,6 +48,13 @@ This module supports the following parameters (see the section `params.modules`
 | `flexsearch.frontmatter`  | false    | If set, includes front matter in the page content. The search index function adds all parameters with the name `content`, `heading`, `title`, `preheading` recursively. |
 | `flexsearch.filter`       | "params" | Restricts the scanned frontmatter variables to the named filter. By default, all front matter variables are scanned. Only applicable when `flexsearch.frontmatter` is set. |
 | `flexsearch.summaryOnly`  | false    | If set, indexes each page's summary instead of its full content. Reduces the size of the generated search index considerably on large sites, at the cost of matching only summary text. |
+| `flexsearch.lazyLoad`     | false    | If set, the search index is emitted as a separate per-language JSON file and fetched on the visitor's first search interaction, instead of being bundled into the core script loaded on every page. See the note below on Content Security Policy. |
+
+> [!NOTE]
+> With `flexsearch.lazyLoad` enabled the search index is fetched at runtime. A
+> site that sets a strict Content Security Policy must allow `connect-src 'self'`.
+> The module declares this directive in its `csp` block, so sites using the
+> Hinode CSP module pick it up automatically.
 
 In addition, the module recognizes the following site parameters (see the section `params.navigation` in `config.toml`):.
 

assets/js/modules/flexsearch/flexsearch.index.js

@@ -1,3 +1,4 @@
+{{- $lazy := site.Params.modules.flexsearch.lazyLoad | default false -}}
 const search = document.querySelector('.search-input')
 const suggestions = document.querySelector('.search-suggestions')
 const background = document.querySelector('.search-background')
@@ -33,57 +34,57 @@ var index = new FlexSearch.Document({
   }
 });
 
+{{ if $lazy -}}
+/*
+  Lazy mode: the index data is fetched from a standalone JSON resource the
+  first time the user interacts with search, instead of being bundled into
+  every page. The data URL is read from the search input's data-search-index
+  attribute (set by the search-input / ModalSearch layouts).
+*/
+let indexStatus = 'idle'; // idle | loading | ready | error
+
+function loadIndex() {
+  if (indexStatus !== 'idle') return;
+  indexStatus = 'loading';
+
+  const url = search.dataset.searchIndex;
+  if (!url) {
+    indexStatus = 'error';
+    console.error('flexsearch: missing data-search-index URL on the search input');
+    return;
+  }
+
+  fetch(url)
+    .then((response) => {
+      if (!response.ok) throw new Error(`HTTP ${response.status}`);
+      return response.json();
+    })
+    .then((docs) => {
+      for (const doc of docs) index.add(doc);
+      indexStatus = 'ready';
+      search.addEventListener('input', showResults, true);
+      // Honor a query typed while the index was still loading.
+      if (search.value) showResults.call(search);
+    })
+    .catch((err) => {
+      indexStatus = 'error';
+      console.error('flexsearch: failed to load search index', err);
+    });
+}
+{{- else -}}
 /*
 Source:
   - https://github.com/nextapps-de/flexsearch#index-documents-field-search
   - https://raw.githack.com/nextapps-de/flexsearch/master/demo/autocomplete.html
 */
 function initIndex() {
-  // https://discourse.gohugo.io/t/range-length-or-last-element/3803/2
-  // Note: pages without a title (such as browserconfig.xml) are excluded
-  {{ $sections := where (where site.Pages "Kind" "section") "Title" "!=" "" }}
-  {{ $list := where site.RegularPages "Title" "!=" "" | append $sections }}
-  {{ $list = where $list ".Params.searchExclude" "!=" true }}
-  {{ $len := (len $list) -}}
-
-  {{ if gt $len 0 }}
-    {{ range $index, $element := sort $list "Title" "asc" -}}
-      {{ $url := .RelPermalink }}
-      {{ if site.Params.modules.flexsearch.canonifyURLs }}{{ $url = .Permalink }}{{ end }}
-      {{ $title := or .Params.indexTitle .LinkTitle .Title }}
-      {{ if gt (strings.RuneCount $title) 33 }}
-          {{ $title = print (substr $title 0 30) "..." }}
-      {{ end }}
-      {{ if and site.Params.main.titleCase (not $element.Params.exact) }}{{ $title = title $title }}{{ end }}
-      {{ $content := "" }}
-      {{ if site.Params.modules.flexsearch.summaryOnly -}}
-        {{ $content = replaceRE "[{}]" "" (.Summary | plainify) }}
-      {{- else -}}
-        {{ $content = replaceRE "[{}]" "" .Plain }}
-      {{- end }}
-      {{ if site.Params.modules.flexsearch.frontmatter }}
-        {{ $key := site.Params.modules.flexsearch.filter | default "params" }}
-        {{ $val := slice }}
-        {{ if ne $key "params" }}{{ $val = index .Params $key }}{{ else }}{{ $val = .Params }}{{ end }}
-        {{ $content = printf "%s %s" (partial "assets/search-meta.html" (dict "key" $key "val" $val)) $content }}
-      {{ end }}
-  index.add({
-    id: {{ $index }},
-    href: "{{ $url }}",
-    title: {{ $title | jsonify }},
-    {{ with .Description -}}
-    description: {{ . | jsonify }},
-    {{- else -}}
-    description: {{ .Summary | plainify | jsonify }},
-    {{- end }}
-    content: {{ trim $content " \r\n" | jsonify }}
-  });
-    {{ end -}}
-  {{ end }}
-
+  {{- range $doc := partial "utilities/GetSearchDocs.html" . }}
+  index.add({{ $doc | jsonify }});
+  {{- end }}
   search.addEventListener('input', showResults, true);
 }
-  
+{{- end }}
+
 function hideSuggestions(e) {
   var isClickInsideElement = suggestions.contains(e.target);
 
@@ -134,7 +135,7 @@ function suggestionFocus(e) {
     focusableSuggestions[nextIndex].focus();
   }
 }
-  
+
 /*
 Source:
   - https://github.com/nextapps-de/flexsearch#index-documents-field-search
@@ -168,7 +169,7 @@ function showResults() {
 
   suggestions.innerHTML = "";
   suggestions.classList.remove('d-none');
-  
+
   // inform user that no results were found
   if (flatResults.size === 0 && searchQuery) {
     const msg = suggestions.dataset.noResults;
@@ -204,17 +205,24 @@ function showResults() {
     if (suggestions.childElementCount == maxResult) break;
   }
 }
-  
+
 if (search !== null && suggestions !== null) {
   document.addEventListener('keydown', inputFocus);
-  document.addEventListener('keydown', suggestionFocus);  
+  document.addEventListener('keydown', suggestionFocus);
   document.addEventListener('click', hideSuggestions);
+  {{ if $lazy -}}
+  search.addEventListener('focus', loadIndex, { once: true });
+  search.addEventListener('click', loadIndex, { once: true });
+  {{- else -}}
   initIndex();
+  {{- end }}
 }
 
 const searchModal = document.getElementById('search-modal')
 if (searchModal !== null) {
   searchModal.addEventListener('shown.bs.modal', function () {
+    {{ if $lazy }}loadIndex();
+    {{ end -}}
     const searchInput = document.getElementById('search-input-modal')
     if (searchInput !== null) {
       searchInput.focus({ focusVisible: true })

config.toml

@@ -1,3 +1,14 @@
+[outputFormats]
+  [outputFormats.searchindex]
+    mediaType = "application/json"
+    baseName = "flexsearch-index"
+    isPlainText = true
+    notAlternative = true
+    permalinkable = true
+
+[outputs]
+  home = ["searchindex"]
+
 [module]
   [module.hugoVersion]
     extended = true
@@ -39,4 +50,7 @@
     frontmatter = false
     filter = ""
     summaryOnly = false
-    localize = true
+    lazyLoad = false
+    localize = true
+  [params.modules.flexsearch.csp]
+    connect-src = ["'self'"]

exampleSite/hugo.toml

@@ -2,6 +2,12 @@ baseURL = 'http://example.org/'
 languageCode = 'en-us'
 title = 'Test site for mod-flexsearch'
 
+[params.modules.flexsearch]
+  lazyLoad = true
+
+[outputs]
+  home = ["HTML", "RSS", "searchindex"]
+
 [module]
   replacements = 'github.com/gethinode/mod-flexsearch/v4 -> ../..'
   [[module.mounts]]

layouts/_partials/assets/search-input.html

@@ -6,10 +6,11 @@
 -->
 
 {{- $class := .class -}}
+{{- $lazy := site.Params.modules.flexsearch.lazyLoad | default false -}}
 
 <div class="d-flex flex-fill ms-md-3{{ with $class }} {{ . }}{{ end }}">
     <form class="search flex-fill position-relative me-auto">
-        <input class="search-input form-control is-search" type="search" placeholder="{{ T "ui_search" }}" aria-label="{{ T "ui_search" }}" autocomplete="off" name="search-input">
+        <input class="search-input form-control is-search" type="search" placeholder="{{ T "ui_search" }}" aria-label="{{ T "ui_search" }}" autocomplete="off" name="search-input"{{ if $lazy }} data-search-index="{{ partialCached "utilities/GetSearchIndex.html" page page.Language.Lang }}"{{ end }}>
         <div class="search-suggestions shadow bg-body rounded d-none" data-no-results="{{ T "ui_no_results" }}"></div>
     </form>
 </div>

layouts/_partials/utilities/GetSearchDocs.html

@@ -0,0 +1,45 @@
+{{- /*
+    Builds the list of documents for the FlexSearch index. Returns a slice of
+    dicts with the keys: id, href, title, description, content.
+
+    This is the single source of truth for index documents — it is consumed
+    both by the eager path (flexsearch.index.js, inline index.add() calls) and
+    the lazy path (flexsearch-index-data.json, fetched on demand).
+
+    Honors the params.modules.flexsearch settings: canonifyURLs, summaryOnly,
+    frontmatter, filter. Pages without a title and pages with searchExclude
+    are omitted.
+*/ -}}
+{{- $sections := where (where site.Pages "Kind" "section") "Title" "!=" "" -}}
+{{- $list := where site.RegularPages "Title" "!=" "" | append $sections -}}
+{{- $list = where $list ".Params.searchExclude" "!=" true -}}
+{{- $docs := slice -}}
+{{- range $index, $element := sort $list "Title" "asc" -}}
+    {{- $url := .RelPermalink -}}
+    {{- if site.Params.modules.flexsearch.canonifyURLs }}{{ $url = .Permalink }}{{ end -}}
+    {{- $title := or .Params.indexTitle .LinkTitle .Title -}}
+    {{- if gt (strings.RuneCount $title) 33 }}{{ $title = print (substr $title 0 30) "..." }}{{ end -}}
+    {{- if and site.Params.main.titleCase (not $element.Params.exact) }}{{ $title = title $title }}{{ end -}}
+    {{- $description := "" -}}
+    {{- with .Description }}{{ $description = . }}{{ else }}{{ $description = $element.Summary | plainify }}{{ end -}}
+    {{- $content := "" -}}
+    {{- if site.Params.modules.flexsearch.summaryOnly -}}
+        {{- $content = replaceRE "[{}]" "" (.Summary | plainify) -}}
+    {{- else -}}
+        {{- $content = replaceRE "[{}]" "" .Plain -}}
+    {{- end -}}
+    {{- if site.Params.modules.flexsearch.frontmatter -}}
+        {{- $key := site.Params.modules.flexsearch.filter | default "params" -}}
+        {{- $val := slice -}}
+        {{- if ne $key "params" }}{{ $val = index .Params $key }}{{ else }}{{ $val = .Params }}{{ end -}}
+        {{- $content = printf "%s %s" (partial "assets/search-meta.html" (dict "key" $key "val" $val)) $content -}}
+    {{- end -}}
+    {{- $docs = $docs | append (dict
+        "id" $index
+        "href" $url
+        "title" $title
+        "description" $description
+        "content" (trim $content " \r\n")
+    ) -}}
+{{- end -}}
+{{- return $docs -}}

layouts/_partials/utilities/GetSearchIndex.html

@@ -0,0 +1,10 @@
+{{- /*
+    Returns the URL of the per-language search index — the `searchindex` output
+    format rendered on the home page. Consumed by the search input layouts to
+    populate the data-search-index attribute the lazy runtime fetches.
+*/ -}}
+{{- $url := "" -}}
+{{- with site.Home.OutputFormats.Get "searchindex" -}}
+    {{- $url = .RelPermalink -}}
+{{- end -}}
+{{- return $url -}}

layouts/_shortcodes/ModalSearch.html

@@ -1,4 +1,5 @@
 {{- $search := partial "utilities/GetSearchConfig.html" (dict "search" true) -}}
+{{- $lazy := site.Params.modules.flexsearch.lazyLoad | default false -}}
 
 {{- if $search.modal }}
     <div id="search-modal" class="modal fade search-modal" tabindex="-1" aria-labelledby="searchModalLabel" aria-hidden="true" aria-modal="true">
@@ -7,7 +8,7 @@
                 <div class="modal-header">
                     <div class="w-100">
                         <form class="search position-relative me-auto">
-                            <input id="search-input-modal" class="search-input form-control is-search" tabindex="1" type="search" placeholder="{{ T "ui_search" }}..." aria-label="{{ T "ui_search" }}" autocomplete="off">
+                            <input id="search-input-modal" class="search-input form-control is-search" tabindex="1" type="search" placeholder="{{ T "ui_search" }}..." aria-label="{{ T "ui_search" }}" autocomplete="off"{{ if $lazy }} data-search-index="{{ partialCached "utilities/GetSearchIndex.html" page page.Language.Lang }}"{{ end }}>
                         </form>
                     </div>
                     <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="{{ T "close" }}"></button>

layouts/index.searchindex.json

@@ -0,0 +1,15 @@
+{{- /*
+    Renders the FlexSearch index documents as a JSON array. Emitted as the
+    `searchindex` output format on the home page so the data is built during a
+    normal page render (where the page context is available) rather than from a
+    detached resource template.
+
+    The lazy runtime fetches this file on the first search interaction. When
+    lazyLoad is disabled the file is still emitted, but empty — the index is
+    bundled into the core script instead.
+*/ -}}
+{{- if site.Params.modules.flexsearch.lazyLoad -}}
+{{- (partial "utilities/GetSearchDocs.html" .) | jsonify -}}
+{{- else -}}
+[]
+{{- end -}}