Repository navigation
See which bots and AI crawlers are visiting your Next.js site #99450
Unanswered
cport1
asked this question in
Show and tell
Replies: 1 comment
|
1. Install the SDK 2. Create the Request Hook (proxy.ts) 3. Add Environment Variable (.env.local) |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
If a crawler downloads your HTML without running JavaScript, your browser analytics script won't see that request. I put together a small Next.js example for observing those requests before the page response.
I build WebDecoy, and this example uses our Next.js adapter. It starts in monitor mode, so a detection doesn't block the request.
For Next.js 16, the request hook lives in
proxy.ts, besideapp(or besidesrc/app):The example pins
@webdecoy/nextjsto0.18.0. Keep the API key server-side. Local checks work without it; reporting detections to the WebDecoy dashboard requires a key. ConfiguretrustProxyfor your actual hosting chain, and integrate this into any existing authentication or redirect flow instead of replacing it.Runnable example · Full walkthrough · SDK source
Extract the example, run
npm ci, copy.env.exampleto.env.local, and runnpm run dev. It listens on port 3107. Add your API key if you want to check dashboard reporting.A couple of things worth checking when you try it:
/.envare different behaviors.Detections aren't a complete page-view log. Compare them with access logs when you need request counts.
I checked the example locally with Next.js 16.3.7: both ordinary requests and the reserved test request returned HTTP 200 in monitor mode. Those checks didn't use a cloud key, so they don't establish dashboard delivery or detection accuracy against real AI crawlers.
Curious how others here handle this: are you checking crawler activity in CDN logs, application logs, or both?
Writing assistance: AI-assisted; implementation and claims reviewed.
All reactions