Skip to content

Migrate /cdn/archive.org JS proxy endpoint to FastAPI#11969

Merged
RayBB merged 6 commits intointernetarchive:masterfrom
bhardwajparth51:migrate-cdn-endpoint-to-fastapi
Mar 1, 2026
Merged

Migrate /cdn/archive.org JS proxy endpoint to FastAPI#11969
RayBB merged 6 commits intointernetarchive:masterfrom
bhardwajparth51:migrate-cdn-endpoint-to-fastapi

Conversation

@bhardwajparth51
Copy link
Contributor

@bhardwajparth51 bhardwajparth51 commented Mar 1, 2026

Closes #11966

Technical

Adds openlibrary/fastapi/cdn.py — a new FastAPI router that mirrors the legacy ia_js_cdn handler in openlibrary/plugins/openlibrary/code.py:

  1. Strict allowlist: only donate.js and athena.js are accepted; all other filenames return 404
  2. Fetches from https://archive.org/includes/{filename} with a 10s timeout
  3. Returns Content-Type: text/javascript and Cache-Control: max-age=86400, matching legacy headers exactly
  4. Router registered in openlibrary/asgi_app.py; existing templates and service worker references (footer.html, donation_banner.html, service-worker-matchers.js) are unaffected

One noted difference from legacy: The original handler doesn't call raise_for_status() — so it silently returns HTTP 200 with an HTML error page when archive.org responds 4xx/5xx. I've returned 502 instead, which feels more correct. Happy to revert if you'd prefer exact parity.

Changes

File Change
openlibrary/fastapi/cdn.py New — FastAPI router
openlibrary/asgi_app.py Register cdn_router
openlibrary/tests/fastapi/test_cdn.py New — unit tests

Requirements Checklist

  • Route with strict filename validation (donate.js|athena.js)
  • Legacy headers matched (Content-Type, Cache-Control)
  • Tests added (mocked, no network calls)
  • Router registered in asgi_app.py
  • Templates and service worker paths unaffected

Testing

docker compose exec web pytest openlibrary/tests/fastapi/test_cdn.py -v
curl -I http://localhost:8080/cdn/archive.org/donate.js   # expect 200, text/javascript, max-age=86400
curl -I http://localhost:8080/cdn/archive.org/evil.js     # expect 404

Stakeholders

@RayBB

@github-project-automation github-project-automation bot moved this to Waiting Review/Merge from Staff in Ray's Project Mar 1, 2026
Copy link
Collaborator

@RayBB RayBB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good just a few docs updates

Copy link
Collaborator

@RayBB RayBB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks prefect!

One small notes for my future self. Maybe we want to caching so we don't hit archive.org so often. But I'll have to ask for the logs to see if it matters.

@RayBB RayBB merged commit 4dc3156 into internetarchive:master Mar 1, 2026
4 checks passed
@github-project-automation github-project-automation bot moved this from Waiting Review/Merge from Staff to Done in Ray's Project Mar 1, 2026
@RayBB RayBB removed the On Testing label Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Migrate /cdn/archive.org JS proxy endpoint from web.py to FastAPI

2 participants