Add Netlify build plugin to cache docforge API responses#957
Add Netlify build plugin to cache docforge API responses#957
Conversation
Deploy preview builds hit GitHub API rate limits because docforge fetches content from 35+ repos on every build. This local Netlify plugin uses the built-in cache utility to persist docforge's HTTP disk cache between deploy preview builds. - Production builds always pull fresh (no cache) - Cache invalidates when any .docforge/*.yaml config changes - Cache expires after 24 hours (TTL)
✅ Deploy Preview for gardener-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughA new Netlify plugin manages Docforge cache artifacts across CI/CD builds by restoring caches before builds and saving them afterward. Supporting configuration updates enable explicit cache directories in the build system and exclude cache files from version control. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 17 minutes and 51 seconds.Comment |
gardener-prow
Bot
added
the
size/M
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@plugins/netlify-plugin-docforge-cache/index.js`:
- Around line 7-15: The collectDigests function should be hardened: replace the
use of readdir(..., {withFileTypes: true, recursive: true}) and entry.parentPath
(Node 22-only) with a manual, synchronous or async recursive directory walk that
builds file paths via path.join(baseDir, relativePath) and only uses entry.name
and entry.isFile(); ensure you skip non-existent directories by wrapping the
top-level call to collectDigests in a try/catch that ignores ENOENT (or let
collectDigests itself catch and return [] on ENOENT). Update references to
collectDigests(...) and the internals of collectDigests to use the manual
recursion and path.join(dir, subpath) construction so Node 20 compatibility and
missing .docforge directory resilience are ensured.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fe2278a0-4329-4778-bb5e-c219ae508a9c
📒 Files selected for processing (5)
.gitignoreMakefilenetlify.tomlplugins/netlify-plugin-docforge-cache/index.jsplugins/netlify-plugin-docforge-cache/manifest.yml
Replace recursive readdir with manual directory walk to avoid dependency on entry.parentPath (Node 21.4+). Also handle ENOENT gracefully when .docforge/ directory doesn't exist.
gardener-prow
Bot
added
the
cla: yes
Production builds should simply ignore the cache, not delete it. The cache is shared across contexts — removing it during prod would force the next deploy preview to rebuild from scratch.
1 participant
/kind enhancement
What this PR does / why we need it:
Deploy preview builds are hitting GitHub API rate limits because docforge fetches documentation content from 35+ repositories on every build. With multiple concurrent deploy previews triggered by PRs, the shared
GITHUB_OAUTH_TOKENexhausts its 5,000 requests/hour budget.This adds a local Netlify build plugin that uses the built-in
utils.cacheutility to persist docforge's HTTP disk cache (httpcache/diskv) between deploy preview builds. Cached responses use ETags for conditional requests —304 Not Modifiedresponses don't count against the rate limit.Key behaviors:
.docforge/**/*.yamlfile changes (viadigests)No changes to docforge itself — uses the existing
--cache-dirflag.Which issue(s) this PR fixes:
Fixes GitHub API rate limiting on Netlify deploy previews.
Special notes for your reviewer:
build.commandinnetlify.tomlmay already be configured via the Netlify dashboard UI — if so, the[build]section here will override itimport/export) matching the repo's"type": "module"in package.jsonSummary by CodeRabbit
New Features
Chores