Add HTTP transport with OAuth, Redis state, and deploy config#183
Merged
Add HTTP transport with OAuth, Redis state, and deploy config#183
Conversation
- http_config.py: HTTP mode setup (auth + no-auth) with OAuth, CSP, middleware - tool_helpers.py: transport-aware helpers (_with_ui, _submission_ui_json, _get_client with per-request JWT, _fetch_task_result) - tool_descriptions.py: transport-specific tool descriptions patched at startup - routes.py: REST endpoints for progress polling and CSV download - result_store.py: Redis-backed result caching with pagination - middleware.py: rate limiting middleware - tools.py: all tools use _with_ui for widget JSON, token storage in Redis - models.py: add input_data/input_json alternatives to input_csv - app.py: separate lifespans for stdio/http/no-auth-http - server.py: CLI args for --http, --no-auth, --host, --port - deploy/: Dockerfile, docker-compose, env example Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… http_config - Remove `settings` field from ServerState; add `everyrow_api_url` and `preview_size` as direct fields with defaults - Add lazy `settings` property that returns the right settings via `_get_http_settings()` / `_get_dev_http_settings()` / `_get_stdio_settings()` - Extract `_BaseHttpSettings` base class for shared Redis and HTTP fields - Remove eager module-level settings singletons from config.py - Inline Redis creation in `configure_http_mode`, extract `_patch_tool_csp` - Update auth.py to use `_get_http_settings()` instead of `http_settings` Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ering - Fix EveryRowAuthProvider constructor to pass (redis, token_verifier) instead of raw settings fields - Use Transport.HTTP/STDIO enum values instead of string literals - Remove duplicate results.html/session.html resource registrations from app.py (only registered in http_config.py for HTTP mode) - Swap middleware order so logging wraps rate limiting (sees all requests) - Merge identical progress descriptions in tool_descriptions.py - Add override notes above everyrow_progress and everyrow_results - Restore manifest.json everyrow_results description for stdio accuracy Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each tool return site now builds the base TextContent list and conditionally prepends the widget JSON only in HTTP mode. This avoids eagerly computing _submission_ui_json (which stores tokens in Redis) in stdio mode where the result was discarded. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All 6 submission tools now call create_tool_response() instead of repeating the stdio/http branching inline. The helper lives in tool_helpers.py alongside _submission_text and _submission_ui_json. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove dead fields: auth_provider (write-only), dev_mode (only read by dead settings property), settings property (never called) - Remove dead _build_csv_url from result_store.py - Add validate_assignment=True to catch invalid transport values - Fix is_http to use == Transport.HTTP (closed-form check) - Fix tests to use Transport.HTTP instead of "streamable-http" string - Use spec=AuthenticatedClient on mock clients for Pydantic compat - Add docstring note about singleton client in no-auth HTTP mode (C1) - Restore field descriptions shortened in models.py (equivalence_relation, merge fields) and fix misleading output_path description for stdio Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The everyrow_client fixture was setting the old app._client attribute which no longer exists after the refactor. Use state.client instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Export assert_stdio_clean from test_stdio_content (rename private helpers to public) and apply it at every submit/progress/results step in test_integration.py. Also assert len(result)==1 and normalize to result[0].text, fixing a stale comment about widget JSON. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the global state.client singleton with a ClientFactory yielded from lifespans and accessed via ctx.request_context.lifespan_context. Tools now receive the client through FastMCP's Context parameter injection instead of reading mutable global state, eliminating 22 patch.object(state, "client", ...) patterns in tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace bare Context with Context[ServerSession, SessionContext] alias, giving type checkers visibility into lifespan_context fields and making the context contract explicit via a dataclass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace make_singleton_client_factory with lambda: client and inline make_http_auth_client_factory into _http_lifespan. Removes indirection that added no value over the inline forms. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The TOKEN_BUDGET setting (default 20k) was defined but unused. This wires it in: after slicing a result page, clamp_page_to_budget binary-searches for the largest subset that fits within the budget (~4 chars/token heuristic). Applied in all three result paths (Redis cache hit, Redis store, inline fallback). The progress completed message now recommends page_size=preview_size and notes the auto-adjustment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…actories ServerState no longer stores everyrow_api_url, preview_size, or token_budget as flat fields. A new `settings` property dispatches to the correct @lru_cache-d factory (_get_stdio_settings, _get_dev_http_settings, or _get_http_settings) based on transport mode. Extracted _CommonSettings base class in config.py so preview_size, token_budget, and everyrow_api_url are defined once and inherited by both StdioSettings and _BaseHttpSettings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
|
@claude code review |
- Convert TaskState from stdlib dataclass to Pydantic BaseModel with PrivateAttr, computed_field properties, and model_dump serialisation - Add write_initial_task_state for submission tools, consolidate file writing into _write_task_state_file - Move progress message building into TaskState.progress_message - Replace ui_dict with model_dump(exclude=_UI_EXCLUDE) at call sites - Remove dead inline-results fallback from everyrow_results (Redis is always available in HTTP mode) - Extract _register_widgets and _register_routes in http_config - Clean up dead re-exports in server.py - Use dedent for multiline tool response strings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… harden edges
- Move Redis client creation and state.mcp_server_url assignment to server.py
(before configure_http_mode) so state is fully initialised before consumers read it
- Pass redis_client explicitly to configure_http_mode, fixing RedisStore/Redis
type mismatch in EveryRowAuthProvider and RateLimitMiddleware
- Add missing lifespan assignment in auth branch of configure_http_mode
- Add ctx argument to all tool calls in test_http_real.py (was missing after
the lifespan context refactor)
- Add everyrow_single_agent to expected tools in test_http_transport.py
- Add meta={"ui": {"resourceUri": ...}} to everyrow_progress tool decorator
- Guard state.store.ping() with None check in both HTTP lifespans
- Reject FAILED/REVOKED tasks in _fetch_task_result before fetching results
- Use get_tool() instead of _tools.get() in _patch_tool_csp
- Use `is not None` in load_csv source counting
- Extract override_state() context manager in conftest.py, deduplicate 4 test fixtures
- Inline _cors_preflight, add return type annotations to route handlers
- Remove dead _validate_redis model validator (lifespan ping catches misconfigured Redis)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove `assert progress["total"] > 0` from screen integration test (screen tasks don't report row-level totals) - Rename _get_http_settings → get_http_settings, _get_dev_http_settings → get_dev_http_settings, _get_stdio_settings → get_stdio_settings (drop leading underscore — these are used across modules) - Extract local `transport` variable in server.py main() Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… external URLs - Fix state.transport never being set (local var instead of state assignment), which caused HTTP mode to run as stdio and exit immediately - Remove progress bar MCP App widget (PROGRESS_HTML, ui://everyrow/progress.html resource, widget JSON from everyrow_progress responses) - Replace "Copy link" button with "Download CSV" button using app.openLink() - Use app.openLink() for all session/download links in results and session widgets so they trigger the host's "Open external link" confirmation dialog - Fix research popover hover: mouseenter → mouseover (mouseenter doesn't bubble) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…collapse - Add linkify() helper to auto-link URLs in cell text and research popovers - Move Download CSV link next to session link as styled anchor - Fix expand/collapse using innerHTML with linkify instead of textContent Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8488aeb to
56529b4
Compare
…de_settings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
request.client.host returns the proxy IP (e.g. Cloudflare tunnel), causing all users to share a single rate limit bucket. Now checks CF-Connecting-IP and X-Forwarded-For before falling back. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…_size in hints - try_cached_result returns None (triggering API re-fetch) when CSV is expired or unreadable, instead of returning an empty preview - Next-page hints use the user's original page_size, not the clamped effective size, so the server re-clamps independently per page - Bump default preview_size from 50 to 100 (token budget auto-clamps) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hide fields that don't apply in each mode: - HTTP: remove input_csv/left_csv/right_csv (server can't read client filesystem) - HTTP: remove output_path from everyrow_results - Stdio: remove offset/page_size from everyrow_results, make output_path required Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ata field Collapse 3 mutually exclusive input fields (input_csv, input_data, input_json) down to 2 (input_csv, data) where data accepts str|list[dict] with auto-detection. MergeInput goes from 6 input fields to 4 (left_data/right_data replace left_input_data/left_input_json/right_input_data/right_input_json). SingleAgentInput.input_data (dict context, not tabular) is unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rgambee
reviewed
Feb 23, 2026
…riptions.py Replace the single ResultsInput model with StdioResultsInput (output_path required) and HttpResultsInput (output_path optional, pagination fields). Each mode gets its own function — everyrow_results_stdio and everyrow_results_http — eliminating runtime schema patching and description overrides. The stdio variant is registered by default; server.py re-registers the HTTP variant when --http is used. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove ARG002 from ruff ignore list. Fix violations: add noqa for auth.py protocol override, remove redundant everyrow_client fixture params from test_integration.py (already pulled in via real_ctx). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the poll token TTL (24h) elapses, get_poll_token() returns None and _get_csv_url() silently interpolated it into the URL, producing a broken download link that 403s. Now _get_csv_url() returns None on expiry, and both callers fall back to a fresh API fetch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rgambee
approved these changes
Feb 23, 2026
Resolve conflicts to integrate forecast and cancel tools from main with the split results functions on this branch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use Redis pipeline with EXPIRE NX to avoid race where TTL is reset on every request, and increase default preview_size to 1000 rows. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ForecastInput now inherits from _SingleSourceInput, gaining the data field so forecast works in HTTP mode where file paths are unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of returning a misleading error when try_store_result fails, return the results inline as a preview with a summary message. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ToolManager.add_tool() is a no-op for existing names, so the HTTP override of everyrow_results was silently ignored, leaving the stdio schema (with required output_path) exposed in HTTP mode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verifies that remove_tool + re-register replaces the stdio schema with HttpResultsInput, preventing regression of the silent no-op bug. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- get_client_ip returns None instead of "unknown" when IP cannot be determined; rate limiter skips these requests instead of grouping them into a shared bucket. - HttpResultsInput no longer validates that output_path's parent directory exists on the server, since in HTTP mode the path comes from a remote client's filesystem. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_client_ip now returns None instead of "unknown", but auth's _client_ip passed it directly to build_key which calls .replace(). Fall back to "unknown" in auth — we want to rate limit unidentified OAuth clients, unlike the middleware where we skip them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
pandas to_dict(orient="records") preserves float('nan') which json.dumps
serializes as NaN — invalid JSON that breaks widget rendering in the client.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment on lines
+134
to
+138
| else: | ||
| summary = ( | ||
| f"Results: showing rows {offset + 1}-{min(offset + page_size, total)} " | ||
| f"of {total} (final page)." | ||
| ) |
There was a problem hiding this comment.
Bug: When offset equals total rows, the pagination summary message becomes nonsensical, e.g., 'showing rows 101-100 of 100'.
Severity: MEDIUM
Suggested Fix
Add a conditional check at the beginning of the else block in _build_result_response. If offset >= total, return a specific message indicating that there are no more rows to show, rather than attempting to calculate and display a row range.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: everyrow-mcp/src/everyrow_mcp/result_store.py#L134-L138
Potential issue: When a user requests a page with an `offset` equal to or greater than
the `total` number of rows, the `_build_result_response` function generates a
nonsensical summary message. For example, if `offset` and `total` are both 100, the
message becomes "showing rows 101-100 of 100 (final page)". While the function correctly
returns an empty set of rows, the accompanying text is confusing for the user. This can
occur because the `offset` parameter in `HttpResultsInput` is only validated to be
non-negative, allowing users to paginate past the end of the dataset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds HTTP (Streamable HTTP) transport mode to the everyrow MCP server, enabling remote deployment with OAuth 2.1 authentication, Redis-backed state, and a Docker deploy stack.
New modules
auth.py— OAuth 2.1 provider (3-leg PKCE: Google → Supabase → MCP Server), JWKS verification, refresh token rotationhttp_config.py— HTTP mode setup: OAuth wiring, custom routes, CSP, middlewareroutes.py— REST endpoints for progress polling and CSV download (bypasses MCP protocol for widgets)middleware.py— Redis-based fixed-window rate limiter per client IPresult_store.py— Redis-backed result caching with pagination and CSV download URLsredis_utils.py— Redis client factory with Sentinel supportconfig.py—HttpSettings,DevHttpSettings,StdioSettingswith_BaseHttpSettingsbase classstate.py—ServerStatewithTransportenum,RedisStore,validate_assignment=Truetool_helpers.py— Transport-aware helpers (_get_client,create_tool_response,_write_task_state)tool_descriptions.py— Per-transport tool description patchingtemplates.py— MCP App widget HTML (progress, results, session)Other changes
everyrow_single_agent— New MCP tool for single-question research (no CSV required)input_data/input_json— All submit tools now accept inline data (not justinput_csv)create_tool_response()— Deduplicated submission return pattern across all 6 submit toolsDockerfile,docker-compose.yaml,.env.exampleStdio impact
equivalence_relation, merge fields,output_path)redis,httpx,PyJWT,pydantic-settings) loaded at module level in both modesArchitecture decisions
@lru_cachegetters), not onServerStateServerStatehasvalidate_assignment=True— rejects invalid transport valuesauth_provider(write-only),dev_mode,settingspropertyis_httpuses closed-form== Transport.HTTPcheck_submission_ui_json()only called in HTTP mode (avoids wasted Redis writes in stdio)Test plan
uv run pytest tests/ -v)RUN_INTEGRATION_TESTS=1)--http --no-auth)🤖 Generated with Claude Code