Search: Add content_last_updated for content-only change tracking#3016
Merged
Search: Add content_last_updated for content-only change tracking#3016
Conversation
Add a new content_last_updated field to DocumentationDocument that only
advances when the page content (stripped_body) actually changes, ignoring
metadata-only changes like navigation reordering or mapping rollovers.
Uses a persistent lookup index (docs-{type}-content-dates-{env}) to
preserve timestamps across index rollovers. Content hashing normalizes
whitespace so reformatting doesn't trigger false updates.
Also updates the sitemap to use content_last_updated instead of
last_updated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use JsonObject instead of raw string literal for index mapping to satisfy the dotnet-format JSON002 rule. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Caution Review failedThe pull request is closed. Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (11)
📝 WalkthroughWalkthroughAdds content-date tracking via a new public ContentDateEnrichment class that provisions and maintains a lookup index, an enrich policy, and an ingest pipeline. DocumentationDocument gains ContentLastUpdated and ContentBodyHash; ContentHash gains CreateNormalized to compute a whitespace-normalized hash. The Elasticsearch exporter computes and assigns ContentBodyHash during export flows and initializes ContentDateEnrichment before publishing synonyms/query rules. On shutdown the exporter completes ingestion orchestration then calls SyncLookupIndexAsync to persist lookup state. ElasticsearchOperations gains a ReindexAsync helper used by the enrichment sync. Sequence DiagramsequenceDiagram
participant Exporter as ElasticsearchMarkdownExporter
participant CDEnrich as ContentDateEnrichment
participant ES as Elasticsearch
participant Ingest as Ingest Pipeline
Exporter->>CDEnrich: InitializeAsync()
CDEnrich->>ES: Ensure lookup index (mappings: url, content_hash, content_last_updated)
CDEnrich->>ES: Create enrich policy (match on url -> content_hash, content_last_updated)
CDEnrich->>ES: Execute enrich policy
CDEnrich->>ES: Create ingest pipeline (set timestamp, enrich, painless script)
CDEnrich-->>Exporter: Ready
Exporter->>Exporter: For each doc: ContentBodyHash = ContentHash.CreateNormalized(doc.StrippedBody)
Exporter->>ES: Index document (includes content_hash)
note over Ingest,ES: On ingestion
Ingest->>ES: Enrich using lookup index (_content_date_lookup)
Ingest->>Ingest: Painless script: if lookup.content_hash == ctx.content_hash then set content_last_updated = lookup.content_last_updated else set content_last_updated = ctx._ingest.timestamp
Ingest->>ES: Finalize indexing with content_last_updated
Exporter->>Exporter: Stop -> Complete orchestrator
Exporter->>CDEnrich: SyncLookupIndexAsync(lexicalAlias)
CDEnrich->>ES: Delete lookup docs (delete-by-query)
CDEnrich->>ES: Reindex lexical -> lookup (uses ReindexAsync)
CDEnrich->>ES: Refresh lookup & re-execute enrich policy
CDEnrich-->>Exporter: Sync complete
Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches✨ Simplify code
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateLookup.cs`:
- Around line 178-179: EscapeJson currently only handles backslashes and quotes
and misses control characters; update the EscapeJson implementation in
ContentDateLookup (the EscapeJson method) to use a proper JSON escaper such as
System.Text.Json: either return JsonSerializer.Serialize(value).Trim('"') or use
JsonEncodedText.Encode(value).ToString() so all control characters and Unicode
are correctly escaped; replace the manual Replace chain with one of these calls
in the EscapeJson method.
- Around line 139-146: The Resolve method uses the non-thread-safe fields
_existing and _changed from ContentDateLookup while being called concurrently;
replace both Dictionary usages with
System.Collections.Concurrent.ConcurrentDictionary instances, update any
TryGetValue/_existing lookups to use ConcurrentDictionary.TryGetValue (or
TryGetValue remains but on the concurrent type), and replace the assignment
_changed[url] = (newContentHash, batchTimestamp) with a thread-safe update
(e.g., TryAdd/AddOrUpdate) so Resolve and any other methods (referencing
_existing, _changed, Resolve) are safe under Parallel.ForEachAsync.
In
`@src/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.cs`:
- Around line 199-210: The current pattern starts _orchestrator.StartAsync(...)
and then awaits loadTask first, risking an unobserved exception on
orchestratorTask if LoadAsync fails; modify the logic in
ElasticsearchMarkdownExporter (around _contentDateLookup.LoadAsync and
_orchestrator.StartAsync) to await both tasks together (e.g., use Task.WhenAll
or equivalent) so exceptions from both tasks are observed/aggregated, then
extract the orchestratorContext from the completed orchestratorTask (or its
result) to log the strategy and aliases.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: ff336ae1-4274-4a37-8ae8-f5e7670f43e4
📒 Files selected for processing (10)
src/Elastic.Documentation/Search/ContentHash.cssrc/Elastic.Documentation/Search/DocumentationDocument.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ContentDateLookup.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.Export.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.cssrc/services/Elastic.Documentation.Assembler/Building/AssemblerBuildService.cssrc/services/Elastic.Documentation.Assembler/Building/EsSitemapReader.cssrc/tooling/docs-builder/Commands/Assembler/AssemblerSitemapCommand.cstests/Elastic.Markdown.Tests/Search/ContentHashTests.cstests/Elastic.Markdown.Tests/Search/DocumentationDocumentSerializationTests.cs
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateLookup.cs
Outdated
Show resolved
Hide resolved
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateLookup.cs
Outdated
Show resolved
Hide resolved
src/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.cs
Outdated
Show resolved
Hide resolved
- Use ConcurrentDictionary for _existing and _changed since Resolve is called from Parallel.ForEachAsync - Use JsonEncodedText.Encode for AOT-safe JSON escaping that handles control characters and Unicode - Use Task.WhenAll to observe both tasks when running lookup load and orchestrator start in parallel Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mpdreamz
requested changes
Apr 2, 2026
Member
Mpdreamz
left a comment
Mpdreamz
left a comment
There was a problem hiding this comment.
Review: content_last_updated via ContentDateLookup
The ContentHash.CreateNormalized addition, the new DocumentationDocument fields, and the unit tests are all good — those should stay regardless.
The main concern is ContentDateLookup: it drains the entire side index into a ConcurrentDictionary on startup, then bulk-upserts changes on shutdown. For the current corpus (~5–10k pages) this works, but it sets a precedent that scales poorly and duplicates infrastructure the project already has.
Hand-rolled PIT pagination
LoadAsync (lines 68–130 of ContentDateLookup.cs) manually implements PIT open / search_after / close — the exact pattern already available as PointInTimeSearch<T> in Elastic.Ingest.Elasticsearch.Helpers. The existing helper handles typed deserialization, automatic PIT lifecycle via IAsyncDisposable, sliced parallel search, and configurable page size / keep-alive / sort / query. The ~80 lines of OpenPitAsync, ClosePitAsync, BuildSearchBody, and the do/while loop could be replaced by roughly:
await using var pit = new PointInTimeSearch<ContentDateEntry>(transport, new PointInTimeSearchOptions
{
Index = _indexName,
Size = 1000
}, jsonOptions);
await foreach (var entry in pit.SearchDocumentsAsync(ct))
_existing[entry.Url] = (entry.ContentHash, entry.ContentLastUpdated);(Same observation applies to the existing EsSitemapReader — separate cleanup.)
Suggested alternative: Elasticsearch Enrich Policy
The project already uses this exact pattern for AI enrichment (AiEnrichmentOrchestrator). The same approach eliminates the in-memory drain entirely by letting Elasticsearch do the join at index time:
- Lookup index —
docs-{type}-content-dates-{env}stores{url, content_hash, content_last_updated}(same as now). - Enrich policy — a
matchpolicy on fieldurl, enriching withcontent_hashandcontent_last_updated. - Ingest pipeline — an enrich processor runs on every document. A script processor compares the incoming
content_hashwith the enriched value: if they differ (or no match exists), it setscontent_last_updatedto_ingest.timestamp; otherwise it preserves the looked-up date. - Post-index save — after indexing, the application already knows every
{url, content_hash, timestamp}it just wrote. Bulk-upsert those into the lookup index (a single bulk call for the whole corpus), then re-execute the enrich policy so the next run sees current data.
This is self-sufficient — no separate reconciliation scan of the main index is needed. The app tracks what it indexed during the normal export flow.
Benefits:
- Zero memory overhead — no startup drain, no
ConcurrentDictionary - Reuses the exact infrastructure patterns from
AiEnrichmentOrchestrator(EnsureLookupIndex, EnsureEnrichPolicy, ExecuteEnrichPolicy, EnsurePipeline) - Scales to any corpus size
- Stale entry cleanup follows the existing
CleanupOrphanedAsyncpattern (PIT-scan lookup, terms-check against main index, delete orphans)
Missing cleanup
The current PR has no mechanism to remove lookup entries for pages that have been deleted or whose URLs changed. Over time the lookup index accumulates orphans. The enrich-policy approach naturally supports cleanup via the same CleanupOrphanedAsync pattern the AI enrichment already uses.
Move content_last_updated resolution from an in-memory drain-and-compare approach to an Elasticsearch enrich policy + ingest pipeline. This eliminates the startup memory overhead of loading the entire lookup index into a ConcurrentDictionary and removes ~200 lines of hand-rolled PIT pagination. The ingest pipeline compares content hashes at index time via an enrich processor and painless script. After indexing, the lookup index is synced via reindex from the lexical index, which also implicitly cleans up orphaned entries for deleted pages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Member
Author
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs (2)
47-56: Consider adding a refresh before policy execution to ensure reindexed documents are visible.After
ReindexToLookupAsynccompletes, the documents may not be immediately searchable due to the default 1-second refresh interval. The subsequentExecutePolicyAsyncmight build the enrich index from stale data.Add explicit refresh
public async Task SyncLookupIndexAsync(string lexicalAlias, Cancel ct) { logger.LogInformation("Syncing content date lookup index {Index} from {Source}", _lookupIndex, lexicalAlias); await DeleteLookupContentsAsync(ct); await ReindexToLookupAsync(lexicalAlias, ct); + await RefreshLookupIndexAsync(ct); await ExecutePolicyAsync(ct); logger.LogInformation("Content date lookup sync complete"); } + +private async Task RefreshLookupIndexAsync(Cancel ct) +{ + await operations.WithRetryAsync( + () => transport.PostAsync<StringResponse>($"{_lookupIndex}/_refresh", PostData.Empty, ct), + $"POST {_lookupIndex}/_refresh", + ct + ); +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs` around lines 47 - 56, SyncLookupIndexAsync may call ExecutePolicyAsync before reindexed documents are visible; after awaiting ReindexToLookupAsync (and before ExecutePolicyAsync) call the Elasticsearch refresh for the target lookup index (the same index referenced by _lookupIndex) and await its completion so newly reindexed documents are searchable; place the refresh call between ReindexToLookupAsync(...) and ExecutePolicyAsync(...) and ensure any client/response errors are handled/logged similarly to DeleteLookupContentsAsync and ReindexToLookupAsync.
58-94: Infrastructure failures are logged but not propagated — documents may silently lackcontent_last_updated.All setup methods (
EnsureLookupIndexAsync,PutEnrichPolicyAsync,ExecutePolicyAsync,PutPipelineAsync) log warnings on failure but don't throw. If the pipeline fails to create, documents will be indexed withoutcontent_last_updatedbeing set, and the issue may go unnoticed until production.If this is intentional (graceful degradation), consider at minimum emitting a diagnostic error via the collector to surface it to CI/CD.
Also applies to: 96-118, 120-132, 134-180
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs` around lines 58 - 94, The setup methods (EnsureLookupIndexAsync, PutEnrichPolicyAsync, ExecutePolicyAsync, PutPipelineAsync) currently only log failures and swallow errors, which can let documents be indexed without content_last_updated; change each method to propagate failures instead of silently continuing: after the transport/operations call check ApiCallDetails.HasSuccessfulStatusCode, and if false either throw a descriptive exception (e.g., InvalidOperationException) including response.ApiCallDetails.DebugInformation and the operation name (HEAD/PUT/POST) or call the existing diagnostics/collector API (e.g., DiagnosticCollector.ReportError) with the same contextual details, then rethrow or return a failed Task; keep the log entry but ensure the error is surfaced to CI by throwing or reporting so callers of EnsureLookupIndexAsync/PutEnrichPolicyAsync/ExecutePolicyAsync/PutPipelineAsync cannot continue as if creation succeeded.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs`:
- Around line 47-56: SyncLookupIndexAsync may call ExecutePolicyAsync before
reindexed documents are visible; after awaiting ReindexToLookupAsync (and before
ExecutePolicyAsync) call the Elasticsearch refresh for the target lookup index
(the same index referenced by _lookupIndex) and await its completion so newly
reindexed documents are searchable; place the refresh call between
ReindexToLookupAsync(...) and ExecutePolicyAsync(...) and ensure any
client/response errors are handled/logged similarly to DeleteLookupContentsAsync
and ReindexToLookupAsync.
- Around line 58-94: The setup methods (EnsureLookupIndexAsync,
PutEnrichPolicyAsync, ExecutePolicyAsync, PutPipelineAsync) currently only log
failures and swallow errors, which can let documents be indexed without
content_last_updated; change each method to propagate failures instead of
silently continuing: after the transport/operations call check
ApiCallDetails.HasSuccessfulStatusCode, and if false either throw a descriptive
exception (e.g., InvalidOperationException) including
response.ApiCallDetails.DebugInformation and the operation name (HEAD/PUT/POST)
or call the existing diagnostics/collector API (e.g.,
DiagnosticCollector.ReportError) with the same contextual details, then rethrow
or return a failed Task; keep the log entry but ensure the error is surfaced to
CI by throwing or reporting so callers of
EnsureLookupIndexAsync/PutEnrichPolicyAsync/ExecutePolicyAsync/PutPipelineAsync
cannot continue as if creation succeeded.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1293e388-205f-4210-b83d-a088d2ee48e0
📒 Files selected for processing (4)
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.Export.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchMarkdownExporter.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchOperations.cs
Throw InvalidOperationException on setup failures (index creation, enrich policy, pipeline) instead of logging warnings and continuing silently. This ensures CI fails fast when infrastructure setup is broken rather than indexing documents without content_last_updated. Add an explicit index refresh between reindex and enrich policy execution in SyncLookupIndexAsync so newly reindexed documents are visible when the policy snapshots the lookup index. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs`:
- Around line 157-168: The script in ContentDateEnrichment currently references
ctx._ingest.timestamp which is not available inside script processors; update
the processor so the script uses metadata().now for the fallback timestamp
(i.e., inside the painless script in the script processor replace
ctx._ingest.timestamp usage with metadata().now), or alternatively remove the
fallback timestamp assignment from the script and add a subsequent set processor
that assigns content_last_updated = {{_ingest.timestamp}} when needed; locate
this change in the JsonObject assigned to the "script" for the script processor
in ContentDateEnrichment and apply one of the two fixes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 176e9e68-c3a8-4c81-a1df-b33c4d5afa90
📒 Files selected for processing (1)
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs
Replace ctx._ingest.timestamp (not available in Painless script
processors) with a set processor using Mustache {{{_ingest.timestamp}}}.
The set processor pre-sets content_last_updated to the ingest timestamp,
and the script processor only overwrites it when the enrich lookup finds
a matching content hash.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs`:
- Around line 207-237: The DeleteLookupContentsAsync and ReindexToLookupAsync
methods currently await operations.DeleteByQueryAsync and
operations.ReindexAsync but ignore the start task result, allowing silent
failures when PostAsyncTaskAsync returns null; update both methods to capture
the returned value (the task id/response) from DeleteByQueryAsync and
ReindexAsync, check if that value is null (or indicates failure), and throw a
descriptive exception (e.g., InvalidOperationException) including context like
the operation name and target index (referencing DeleteLookupContentsAsync,
ReindexToLookupAsync, DeleteByQueryAsync, ReindexAsync and the
PostAsyncTaskAsync result/taskId) so callers can detect and handle failed start
of the async operation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: c810e016-be94-4171-a2ef-4d1e83c9f05e
📒 Files selected for processing (1)
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs
DeleteByQueryAsync, ReindexAsync, and UpdateByQueryAsync now throw InvalidOperationException when PostAsyncTaskAsync returns null instead of silently skipping the poll. These are wait-for-completion methods where callers expect the operation to have succeeded on return. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mpdreamz
approved these changes
Apr 8, 2026
Contributor
ChatThere are 3 ways to chat with CodeRabbit:
CodeRabbit commands
Other keywords and placeholders
Status, support, documentation and community
|
Member
Author
|
@coderabbitai review |
Contributor
✅ Actions performedReview triggered.
|
Member
Author
|
@coderabbitai full review |
Contributor
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs`:
- Around line 51-52: The current sequence calls DeleteLookupContentsAsync which
removes the only durable lookup before ReindexToLookupAsync writes a
replacement; instead, change the flow to reindex into a new staging index (e.g.,
derive a staging name from lexicalAlias), run the full copy and refresh against
that staging index, verify success, then atomically swap the lexicalAlias to
point to the new index and only then delete the old lookup; update usages in
InitializeAsync to continue referencing the alias so the swap is transparent and
ensure ReindexToLookupAsync and DeleteLookupContentsAsync are replaced or
refactored to support staging + alias-swap semantics.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: eee48ab1-51b6-48f2-adfa-2f64d4511ffb
📒 Files selected for processing (4)
src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cssrc/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchOperations.cssrc/services/Elastic.Documentation.Assembler/Building/AssemblerBuildService.cssrc/tooling/docs-builder/Commands/Assembler/AssemblerSitemapCommand.cs
✅ Files skipped from review due to trivial changes (2)
- src/services/Elastic.Documentation.Assembler/Building/AssemblerBuildService.cs
- src/tooling/docs-builder/Commands/Assembler/AssemblerSitemapCommand.cs
🚧 Files skipped from review as they are similar to previous changes (1)
- src/Elastic.Markdown/Exporters/Elasticsearch/ElasticsearchOperations.cs
Comment on lines
+51
to
+52
| await DeleteLookupContentsAsync(ct); | ||
| await ReindexToLookupAsync(lexicalAlias, ct); |
Contributor
There was a problem hiding this comment.
Don’t rebuild the persisted lookup in place.
Line 51 deletes the only durable content-date state before Line 52 has produced a replacement. If shutdown is cancelled or the reindex fails partway through, the lookup index stays empty/partial, and the next InitializeAsync will execute the enrich policy against that truncated data set. Unchanged pages will then get a fresh content_last_updated. Rebuild into a staging index and swap it in only after the copy and refresh succeed.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/Elastic.Markdown/Exporters/Elasticsearch/ContentDateEnrichment.cs` around
lines 51 - 52, The current sequence calls DeleteLookupContentsAsync which
removes the only durable lookup before ReindexToLookupAsync writes a
replacement; instead, change the flow to reindex into a new staging index (e.g.,
derive a staging name from lexicalAlias), run the full copy and refresh against
that staging index, verify success, then atomically swap the lexicalAlias to
point to the new index and only then delete the old lookup; update usages in
InitializeAsync to continue referencing the alias so the swap is transparent and
ensure ReindexToLookupAsync and DeleteLookupContentsAsync are replaced or
refactored to support staging + alias-swap semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add a
content_last_updatedfield toDocumentationDocumentthat only advances when page content (stripped_body) actually changes, as opposed tolast_updatedwhich advances on any change (metadata, navigation, mappings).Why
The sitemap and search features need to know when content was last meaningfully updated, not when metadata like navigation ordering or synonym configs changed.
last_updatedconflates content and metadata changes.How
content_hashfield: whitespace-normalized SHA-256 hash ofstripped_body(survives reformatting)content_last_updatedfield: only advances whencontent_hashchangesdocs-{type}-content-dates-{env}) stores{url, content_hash, content_last_updated}— survives index rollovers since it's a fixed-name index independent of the document index lifecyclecontent_last_updatedinstead oflast_updatedTest plan
ContentHash.CreateNormalizedunit tests (whitespace normalization)DocumentationDocumentserialization round-trip test with new fieldscontent_last_updatedunchanged; modify a page → it advances; modify only navigation → it stays the same🤖 Generated with Claude Code