wrangler dev fetch service from localhost

[vijaymayekar]

🐛 Bug Report

I am testing workers locally. Worker fetches json from one of the service, when the service endpoint is of aws it works fine, but when i switch endpoint to localhost:5000 it stops working and throws error The Cloudflare domain resolves to a local or disallowed IP address or an IP address not associated with the domain.

Environment

• operating system:
• output of rustc -V:
• output of node -v: 12.18.3
• output of wrangler -V: 1.12.2
• contents of wrangler.toml

name = "i18n-example"
type = "webpack"
account_id = "XXXX"
workers_dev = true
route = ""
zone_id = ""

[site]
bucket = "./public"
entry-point = "workers-site"

Steps to reproduce

Use code from https://developers.cloudflare.com/workers/examples/fetch-html and replace someHost with any localhost website.

What did you expect to see?

Loaded website with all the content

What did you see instead?

Error 1003 Access Denied: Direct IP Access Not Allowed

[jbampton]

When you run wrangler dev the code actually runs on the Cloudflare network and not on your local machine.

So the Cloudflare network would have access to AWS but not localhost.

cloudflare/cloudflare-docs#10

https://blog.cloudflare.com/announcing-wrangler-dev-the-edge-on-localhost/

[vijaymayekar]

@jbampton Thanks for the quick reply.

Is there any workaround to test this using localhost service?

[Gunner245]

I was able to work around this by exposing my localhost to the internet using localtunnel:
https://github.com/localtunnel/localtunnel. Do:

> lt --port 80  
your url is: https://mr-swan-62.loca.lt

I then put a setting on dev in my wrangler.toml to set the url localhost is exposed on, and then a bit of logic in the worker to say if the url is set to reroute requests via this:

if(DEV_INTRTCEPT_HOMEPAGE){
    const requestChanges = {
        headers: { "Bypass-Tunnel-Reminder": "true"}
    };
    var url = new URL(request.url);
    request = new Request(
        `${DEV_INTRTCEPT_HOMEPAGE}/${url.pathname}`,
        new Request(request, requestChanges),
    )
  }

[bryanvaz]

We need this functionality as well. Right now we're about 1 week into using Workers and trying to develop a cloudflare worker in front of an origin server that would move logic to the edge, and takes about 60-70% longer than it should. We waste more time waiting for a new version of the origin server deploy, in order to test the tiniest change on either side, than we actually spend developing our Worker.

I get the reasoning behind wanting the 'dev' version to run on cloudflare servers in order to eliminate any inconsistencies between a local simulated runtime, but that's what the whole point of nightly, SIT, and UAT builds. It essentially precludes the use of the fetch and sub-request system as an integral, non-overlapping part of a wholistic solution, and relegates any potential uses of Workers to a "nice-to-have", but not a mission-critical or mandatory part of a solution's architecture.

Even in the Starter example Apollo GraphQL Server, which is probably one of the best examples of building a Worker that provides actual business value-add by moving logic to edge, has sub-requests and a fetch back to the origin server. This is an architecture that would have serious business benefits to anyone, specifically, by moving light compute tasks to the CDN edge along with caching of frequent data, while the origin handled more complex tasks such as analytics with a large database, you could dramatically lower origin costs (our main use case.) However, if you were actively developing this GraphQL Starter example with another Apollo origin server at the same time, you would have to redeploy any new change to the Apollo origin server in order to test with the Worker that sits in front of it.

This is not a new request either: #901 #1095, #1093, #1047

If it wasn't for the Workers KV and Durable Objects (and its support for WebSockets), we'd go back to Lambda@Edge in a heartbeat (or if AWS were to launch a similar solution.) At least with Lambda@Edge we can use serverless-offline which we currently use for the IoT side of our stack.

Even a simple docker for Workers that could run in a CI pipeline and on dev machines would be better than wonky system of localtunnels or temporary dev deployments to the public internet.

[jacob]

While I have not experienced this exact issue I was also surprised to find that wrangler dev does not run the code locally. That is not what I expected coming from using other NodeJS at the edge solutions like Google Cloud Functions and Lambda Edge. Retroactively it explains why using wrangler dev with a nodejs worker feels like an inferior development experience compared to other similar products. For example the debug support for diagnosing exceptions is not a pleasant development experience and I have spent a lot more time frustrated while working on methods for Cloudflare than for other similar products. At first I thought that the poor experience was due to wrangler being written in Rust and not native NodeJS but now I realize I was probably getting tripped up on the unexpected development behavior. I am writing this comment to advocate for adding a local execution mode for development since I believe that will significantly improve the developer productivity with Cloud Workers, even if it does not cover 100% of the functionality available in the deployment target environment. The issues that I was unable to easily see were simple issues with modifying requests while proxying that having viewable exception backtraces and reliable fast code updates to the dev instance would have made much much quicker to resolve. I routinely restarted wrangler because it was running stale code and I thought that would fix it, now I see that is a result of the remote code dev setup causing poor latency for the development cycle of test, change, test.

[grahamlyons]

I would also like to have a solution to run Cloudflare Workers locally. It wasn't clear to me that wrangler dev does not run locally and, like the original issue, I was trying to fetch from localhost to run acceptance tests locally and in an a CD pipeline.

This use case is one that I've used before with AWS Lambda (using Sam or Serverless) and is great for testing functions by mocking their dependent services.

What is the plan for addressing this?

[jgentes]

@grahamlyons I'd be curious to know if Miniflare will suit your needs for this. https://miniflare.dev/

[grahamlyons]

@grahamlyons I'd be curious to know if Miniflare will suit your needs for this. https://miniflare.dev/

Yes, yes! A thousand times, yes! This looks like it is exactly what I want wrangler dev to do. Thank you so much. Can you tell me if this it something that Cloudflare are involved with or will support?

[jgentes]

Stay tuned 😉

[KevinColemanInc]

For ppl trying to get miniflare to work, you need to set the upstream value to the host's url (of your other app), this can be done either in wrangler.toml as below or pass in to the cli via --upstream.

[miniflare]
upstream = "http://localhost:3000/" ## --upstream

[penalosa]

Thanks for suggesting this feature! Wrangler v1 is now deprecated and support is only being provided for critical updates or security concerns. As such, we are closing this issue.

New versions of Wrangler are maintained in the workers-sdk repo. If this is a feature you’d like to see in wrangler v2+, please feel free to report it in the workers-sdk repo. For more info about wrangler v1 deprecation, please check out our blog post.

To get this feature in Wrangler v2+, use npx wrangler dev --local