Meteor server exception when mistakenly used as a proxy

[fortress-of-solitude]

Even with no clients running the server throws this exception at random times but consistently. Strangely the apparently offending URL is always the same as shown below.

Error: url must be a relative URL: http://www.baidu.com/
at _.extend.classify (app/packages/routepolicy/routepolicy.js:92:13)
at appUrl (/root/application/.meteor/local/build/server/server.js:160:41)
at Object.run as handle
at next (/root/.meteor/tools/cc18dfef9e/lib/node_modules/connect/lib/proto.js:190:15)
at Router._start (app/packages/router/lib/router_server.js:83:20)

[awwx]

I think this would only happen if a client is making a GET request with a absolute URL instead of a relative URL, which is usually how requests are made to proxies instead of to end servers. This means that someone is connecting to your server as a client, and suggests that they are attempting to use your server as a proxy.

Perhaps some malware is scanning your network looking for open proxies.

[fortress-of-solitude]

I agree. Moved to another less well know port and the problem has not reoccurred. I think it is safe to close this one.

[awwx]

Actually I'd leave this issue open, and rename the issue title to "Meteor server exception when mistakenly used as a proxy"

Ideally Meteor should detect this case and do something sensible (such as returning an error to the client), instead of throwing an exception. (A robust server is able to handle being used incorrectly).

Reproduction is easy: configure your browser's proxy to point to a Meteor app (such as localhost:3000)

and open any web page (www.google.com).

=> Meteor server running on: http://localhost:3000/
Error: url must be a relative URL: http://www.google.com/
    at _.extend.classify (app/packages/routepolicy/routepolicy.js:92:13)
    at appUrl (/home/andrew/proxy/.meteor/local/build/server/server.js:160:41)
    ...

[fortress-of-solitude]

Reopening and re-titling based on new reproduction steps and a desire for a more robust server handling for such events.

[glasser]

I agree, we shouldn't log a stack trace and should probably return something like a 4xx instead of a 500 to the user.

[maxharris9]

Is anything happening on this front? I am experiencing this exact problem.

[crapthings]

i've got this too, and site stop running.

[waiholiu]

I think our site has been falling over randomly due to this issue. Is there a fix for this yet?

[Tarang]

There are all these bots looking for open proxies doing this e.g

Error: url must be a relative URL: http://hotel.qunar.com/render/hoteldiv.jsp?&__jscallback=XQScript_4
     at _.extend.classify (packages/routepolicy/routepolicy.js:103)
     at appUrl (packages/webapp/webapp_server.js:153)
     at Object.handle (packages/webapp/webapp_server.js:486)
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at Object.Package [as handle] (packages/app-bitcoin-payments-core/inbound_callback.
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/

Is it possible these don't come up spamming the logs?

[glasser]

Pull requests welcome here.

[threehex]

We have this issue on Digital Ocean, site stopped running. Any solution?

[ghost]

Same issue on Digital Ocean. Site is running, but I cannot deploy updates. http://stackoverflow.com/questions/27031100/bots-preventing-meteor-server-from-deploying-on-digital-ocean-with-meteor-up/27031891#27031891

[JeremySaks]

@fortress-of-solitude what is the less well known port you selected to hide your app from these scans?

And is this the only bridge fix until #2393 or another solution can be merged? I can second what others are experiencing, Error: url must be a relative URL not only crashes my deployment but prevents redeployment (Digital Ocean + Meteor Up) so it is especially pernicious.

[glasser]

fyi, #2393 is in a queue of bugs I'm hoping to churn through soon. Not until after I get the next (tool performance focused) release out though!

[ghost]

I'm questioning whether this is the actual cause of failed deployments. Run mup logs -f in one terminal while running mup deploy in the other terminal. At least for me, the errors caused by the bots did not occur during deployment; they occurred previously while the app was running successfully. More detailed discussion here: http://stackoverflow.com/questions/27031100/bots-preventing-meteor-server-from-deploying-on-digital-ocean-with-meteor-up/27031891#27031891

[benjyz]

Have this one, too. Is this specific to Digitalocean?

Error: url must be a relative URL: http://proxyjudge.us/

@JeremySaks mup specific problems are something to be filed upstream. You can try demeteorizer - perhaps that works for you.

[ghost]

@benjyz that error is saying that a bot associated with proxyjudge.us is trying to crawl your site. Has nothing to do with DigitalOcean, though it's possible the bots are targeting DigitalOcean servers specifically.

[nooitaf]

I run multiple mup-deployed sites on digital ocean and i see those errors in my logs too, but there is no reason for those errors to be related to deployment-errors. I can deploy without any problems. Meteor is a node process, executed by forever and that gets killed (by root) if your mup deploy upload succeeds. Most time when my deploy fails its related to conflicting versions in mup, node or mongodb.

Still, would like this error to be more silent.

[JeremySaks]

After looking more closely at the logs I agree that the errors are not related to deployment.

However, in addition to crashing the running process, the errors apparently cause appcache to try to serve assets from the app's earlier crashed state, including after page refreshes. So the end user is still experiencing a crashed app even while Meteor is running following a successful redeployment, which is why I and others thought it was an error in redeployment.

Manually removing the app from the browser manifest (e.g. //appcache-internals in Chrome) fixes things (but of course that's not workable for sites with active users).

[glasser]

Fixed.

[terenceng2010]

Our team faces a similar issue when using fast render:

Error: url must be a relative URL: http://51.254.206.142/httptest.php
at [object Object]..extend.classify (packages/routepolicy/routepolicy.js:107:1)
at IsAppUrl (packages/meteorhacks_fast-render/packages/meteorhacks_fast-render.js:138:1)
at [object Object].filterFunction (packages/meteorhacks_fast-render/packages/meteorhacks_fast-render.js:168:1)
at [object Object].PickerImp._dispatch (packages/meteorhacks_picker/packages/meteorhacks_picker.js:44:1)
at processNextSubRouter (packages/meteorhacks_picker/packages/meteorhacks_picker.js:80:1)
at processNextRoute (packages/meteorhacks_picker/packages/meteorhacks_picker.js:73:1)
at processNextMiddleware (packages/meteorhacks_picker/packages/meteorhacks_picker.js:56:1)
....

https://github.com/kadirahq/fast-render/blob/c85e49f08eb0321a0f16ff1fcda8efe80f935e25/lib/server/utils.js
https://github.com/meteor/meteor/blob/dc3cd6eb92f2bdd1bb44000cdd6abd1e5d0285b1/packages/routepolicy/routepolicy.js

As you can see, in fastrender's isAppUrl calls routepolicy.classify, but classify has its own if (url.charAt(0) !== '/') checking that throw the error, it is suitable to modify classify so it calls routepolicy.isValidUrl instead? Or you think there are some changes needed on fast render?

Thanks.

[kaushik1979]

I updated the meteor version also but I am still getting this error. What am I supposed to do to resolve this error?

[elie222]

I also still see errors like this

[ppryde]

Yup me too :(

[dovrosenberg]

Here too.

[hellified]

Got it here too. I'm wondering ... is this related to running the app in a docker container and mapping the port to the host? Is that seen as forwarding by meteor?

[nerdvibe]

Got it as well... I'm so afraid that this might affect SEO.

By the way my configuration:

• ssr with blaze
• Service Workers
• mup
• https only

errors:
Error: url must be a relative URL: http://www.google.com/
Error: url must be a relative URL: http://www.baidu.com/cache/global/img/gs.gif
Error: url must be a relative URL: http://www.baidu.com/favicon.ico
Error: url must be a relative URL: http://www.bing.com/
Error: url must be a relative URL: http://httpheader.net/