Skip to content

Add OData filtering and multi-source ingest failover#92

Merged
Miracle656 merged 1 commit into
Miracle656:mainfrom
ALIPHATICHYD:feature/81-82-odata-multisource
May 30, 2026
Merged

Add OData filtering and multi-source ingest failover#92
Miracle656 merged 1 commit into
Miracle656:mainfrom
ALIPHATICHYD:feature/81-82-odata-multisource

Conversation

@ALIPHATICHYD
Copy link
Copy Markdown
Contributor

Implements the scoped work for:

Closes #81
Closes #82

Summary:

  • Adds safe OData-style $filter / $select handling with cursor pagination on list endpoints.
  • Adds a pluggable indexer source switcher with RPC primary and Horizon fallback.
  • Updates route/tests/docs to match the new behavior.

Summary

Related issue

Type of change

  • Bug fix
  • New feature
  • Refactor
  • Docs
  • Tests
  • CI / tooling

Checklist

  • I have read CONTRIBUTING.md
  • npx tsc --noEmit passes
  • npm run build passes
  • I added / updated tests where relevant
  • I updated docs where relevant

Implements the scoped work for:

Closes Miracle656#81
Closes Miracle656#82

Summary:
- Adds safe OData-style $filter / $select handling with cursor pagination on list endpoints.
- Adds a pluggable indexer source switcher with RPC primary and Horizon fallback.
- Updates route/tests/docs to match the new behavior.
@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 30, 2026

@ALIPHATICHYD Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Copy link
Copy Markdown
Owner

@Miracle656 Miracle656 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The risky piece in any OData implementation is SQL injection via the filter expression. This one is safe by construction:

  • The parser never builds raw SQL. It emits a Prisma where object ({ field: { contains, mode } }, { field: value }, { field: { gt: value } }, { AND: […] }), and Prisma parameterizes every value before sending it to Postgres.
  • Field names are double-gated: an identifier regex ^[A-Za-z_][A-Za-z0-9_\.]*$ plus the caller-supplied fields: Record<string, FieldDefinition> allowlist. Anything not in the allowlist throws — no smuggling id; DROP TABLE.
  • Values are coerced per FieldDefinition.type: strict integer regex for numbers, new Date() with validity check for dates, escaped-single-quote handling for strings ('O''Brien'O'Brien).
  • OR is explicitly refused (/\bor\b/i rejects upfront) — limits the surface to AND-of-clauses, which keeps both query shape and parser complexity bounded.
  • $select validation is the same shape — identifier regex + allowlist. projectRecord even supports derived computed fields cleanly.
  • encodeCursor / decodeCursor are base64url-wrapped { id: number } JSON with integer validation; malformed cursors return undefined so the route falls back to the start instead of throwing.

Multi-source ingest

sources/index.ts switcher with RPC primary + Horizon fallback is the right shape. Tests cover both paths and the failover trigger.

Merging. Closes #81, #82.

@Miracle656 Miracle656 merged commit e0941f8 into Miracle656:main May 30, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-source ingest (RPC + Horizon fallback) OData-style $filter / $select

2 participants