Conversation
ReadOnlyRequestFilter blocked write requests to /graph/v2/service/sys/label/info/edge which is called internally during app startup (warmup) with POST/PUT/GET/DELETE methods for system metadata initialization. This caused the app to fail readiness probe in read-only mode (shadow tenants). Allow the exact system init path while keeping all other write operations blocked. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
7dc1421 to
23ea163
Compare
|
@em3s Could you review this? This fixes a startup failure when The fix adds an exact-path exception for this single warmup endpoint. Pod logs from a read-only deployment: |
Warmup (ServiceLabelEdgeControllerWarmUp) sends POST/PUT/DELETE to /graph/v2/service/sys/label/info/edge during startup. This test verifies the server boots successfully when both read-only mode and warmup are enabled — the scenario that was failing in production. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
|
Good workaround, but can we solve it differently? For example, enabling ReadOnlyRequestFilter only after the warmup? |
|
@em3s Thanks for the suggestion — agreed, activating the filter after warmup is a cleaner approach. I'll update the PR. |
… exception Instead of allowlisting the system init path, the filter now starts inactive and is activated after warmup completes: - ReadOnlyRequestFilter: starts inactive, activate() enables blocking - ServiceLabelEdgeControllerWarmUp: calls activate() after readiness UP - WebFilterConfig: activates on ApplicationReadyEvent when warmup is disabled - Tests updated: verify inactive/activated behavior, remove path exception Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
|
Updated — the filter now starts inactive and is activated after warmup completes, instead of allowlisting a specific path. Changes:
|
|
Activation flow & safety:
|
- Reorder: default → mutation mode → read-only off/on → warmup combinations - Add StartUpWithReadOnlyAndWarmUpDisabledTest (warmup=false + read-only=true) - Rename StartUpWithReadOnlyAndWarmUpTest → StartUpWithReadOnlyAndWarmUpEnabledTest Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
|
@em3s Gentle reminder — updated per your feedback (filter activates after warmup instead of path allowlist). |
|
How about just adding some condition to avoid writing? @eazyhozy . |
|
@em3s Thanks for the suggestion — it led me to reconsider the approach. Since read-only servers don't need write warmup, disabling warmup via |
…ctivation - ServiceLabelEdgeControllerWarmUp: use GET-only warmup when readOnly=true - ReadOnlyRequestFilter: remove activate()/AtomicBoolean, active from startup - WebFilterConfig: remove readOnlyFilterActivator bean
|
@em3s Updated the PR based on your feedback. Problem: When running with Minimal fix in this PR:
Regarding |
…warmup split) Simpler fix incoming — skip write warmup via readOnly condition instead of restructuring warmup flow. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GraphConfig: add readOnly field - Graph.kt: extend existing label.entity.readOnly check with || readOnly - GraphConfiguration: pass serverProperties.readOnly to GraphConfig - ServiceLabelEdgeControllerWarmUp: use GET-only methods when readOnly - StartUpTest: add E2E test for read-only + warmup enabled combination Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fa3254c to
f7f0c2a
Compare
|
Pushed a commit based on my earlier suggestion — extending the |
|
@em3s Thanks for pushing the Graph.kt change — this covers the engine-level warmup I was unsure about scoping in. LGTM, updated the PR description to reflect the final approach. |
Summary
Fix read-only mode warmup: skip write operations at both HTTP and engine levels instead of blocking them with the filter.
Problem
When running with
actionbase.read-only=trueand warmup enabled, two write paths caused issues:ServiceLabelEdgeControllerWarmUp): Sends POST/PUT/DELETE to/graph/v2/service/sys/label/info/edgeon localhost — blocked byReadOnlyRequestFilterGraph.warmUp): Performs fake edge writes to warm HBase connections — unnecessary write load on production HBase from read-only serversChanges
ServiceLabelEdgeControllerWarmUp: Use GET-only warmup methods whenreadOnly=trueGraphConfig: AddreadOnly: BooleanfieldGraphConfiguration: PassserverProperties.readOnlytoGraphConfigGraph.kt: Extend existinglabel.entity.readOnlycheck with|| readOnlyto skip engine-level write warmupStartUpTest: Add E2E test forread-only=true+warmup.enabled=truecombinationHow to Test