New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meteor server exception when mistakenly used as a proxy #1212

Closed
fortress-of-solitude opened this Issue Jul 9, 2013 · 27 comments

Comments

Projects
None yet
@fortress-of-solitude

fortress-of-solitude commented Jul 9, 2013

Even with no clients running the server throws this exception at random times but consistently. Strangely the apparently offending URL is always the same as shown below.

Error: url must be a relative URL: http://www.baidu.com/
at _.extend.classify (app/packages/routepolicy/routepolicy.js:92:13)
at appUrl (/root/application/.meteor/local/build/server/server.js:160:41)
at Object.run as handle
at next (/root/.meteor/tools/cc18dfef9e/lib/node_modules/connect/lib/proto.js:190:15)
at Router._start (app/packages/router/lib/router_server.js:83:20)

@awwx

This comment has been minimized.

Contributor

awwx commented Jul 9, 2013

I think this would only happen if a client is making a GET request with a absolute URL instead of a relative URL, which is usually how requests are made to proxies instead of to end servers. This means that someone is connecting to your server as a client, and suggests that they are attempting to use your server as a proxy.

Perhaps some malware is scanning your network looking for open proxies.

@fortress-of-solitude

This comment has been minimized.

fortress-of-solitude commented Jul 10, 2013

I agree. Moved to another less well know port and the problem has not reoccurred. I think it is safe to close this one.

@awwx

This comment has been minimized.

Contributor

awwx commented Jul 10, 2013

Actually I'd leave this issue open, and rename the issue title to "Meteor server exception when mistakenly used as a proxy"

Ideally Meteor should detect this case and do something sensible (such as returning an error to the client), instead of throwing an exception. (A robust server is able to handle being used incorrectly).

Reproduction is easy: configure your browser's proxy to point to a Meteor app (such as localhost:3000)

and open any web page (www.google.com).

=> Meteor server running on: http://localhost:3000/
Error: url must be a relative URL: http://www.google.com/
    at _.extend.classify (app/packages/routepolicy/routepolicy.js:92:13)
    at appUrl (/home/andrew/proxy/.meteor/local/build/server/server.js:160:41)
    ...
@fortress-of-solitude

This comment has been minimized.

fortress-of-solitude commented Jul 10, 2013

Reopening and re-titling based on new reproduction steps and a desire for a more robust server handling for such events.

@glasser

This comment has been minimized.

Member

glasser commented Jul 16, 2013

I agree, we shouldn't log a stack trace and should probably return something like a 4xx instead of a 500 to the user.

@maxharris9

This comment has been minimized.

maxharris9 commented Dec 9, 2013

Is anything happening on this front? I am experiencing this exact problem.

@crapthings

This comment has been minimized.

crapthings commented Dec 13, 2013

i've got this too, and site stop running.

@waiholiu

This comment has been minimized.

waiholiu commented Jul 30, 2014

I think our site has been falling over randomly due to this issue. Is there a fix for this yet?

@Tarang

This comment has been minimized.

Contributor

Tarang commented Jul 30, 2014

There are all these bots looking for open proxies doing this e.g

Error: url must be a relative URL: http://hotel.qunar.com/render/hoteldiv.jsp?&__jscallback=XQScript_4
     at _.extend.classify (packages/routepolicy/routepolicy.js:103)
     at appUrl (packages/webapp/webapp_server.js:153)
     at Object.handle (packages/webapp/webapp_server.js:486)
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/
     at Object.Package [as handle] (packages/app-bitcoin-payments-core/inbound_callback.
     at next (/root/apps/app-core/programs/server/npm/webapp/main/node_modules/connect/lib/

Is it possible these don't come up spamming the logs?

@glasser

This comment has been minimized.

Member

glasser commented Jul 31, 2014

Pull requests welcome here.

@threehex

This comment has been minimized.

threehex commented Aug 10, 2014

We have this issue on Digital Ocean, site stopped running. Any solution?

@ghost

This comment has been minimized.

ghost commented Nov 20, 2014

@JeremySaks

This comment has been minimized.

JeremySaks commented Dec 12, 2014

@fortress-of-solitude what is the less well known port you selected to hide your app from these scans?

And is this the only bridge fix until #2393 or another solution can be merged? I can second what others are experiencing, Error: url must be a relative URL not only crashes my deployment but prevents redeployment (Digital Ocean + Meteor Up) so it is especially pernicious.

@glasser

This comment has been minimized.

Member

glasser commented Dec 12, 2014

fyi, #2393 is in a queue of bugs I'm hoping to churn through soon. Not until after I get the next (tool performance focused) release out though!

@ghost

This comment has been minimized.

ghost commented Dec 13, 2014

I'm questioning whether this is the actual cause of failed deployments. Run mup logs -f in one terminal while running mup deploy in the other terminal. At least for me, the errors caused by the bots did not occur during deployment; they occurred previously while the app was running successfully. More detailed discussion here: http://stackoverflow.com/questions/27031100/bots-preventing-meteor-server-from-deploying-on-digital-ocean-with-meteor-up/27031891#27031891

@benjyz

This comment has been minimized.

benjyz commented Dec 13, 2014

Have this one, too. Is this specific to Digitalocean?

Error: url must be a relative URL: http://proxyjudge.us/

@JeremySaks mup specific problems are something to be filed upstream. You can try demeteorizer - perhaps that works for you.

@ghost

This comment has been minimized.

ghost commented Dec 13, 2014

@benjyz that error is saying that a bot associated with proxyjudge.us is trying to crawl your site. Has nothing to do with DigitalOcean, though it's possible the bots are targeting DigitalOcean servers specifically.

@nooitaf

This comment has been minimized.

nooitaf commented Dec 13, 2014

I run multiple mup-deployed sites on digital ocean and i see those errors in my logs too, but there is no reason for those errors to be related to deployment-errors. I can deploy without any problems. Meteor is a node process, executed by forever and that gets killed (by root) if your mup deploy upload succeeds. Most time when my deploy fails its related to conflicting versions in mup, node or mongodb.

Still, would like this error to be more silent.

@JeremySaks

This comment has been minimized.

JeremySaks commented Dec 15, 2014

After looking more closely at the logs I agree that the errors are not related to deployment.

However, in addition to crashing the running process, the errors apparently cause appcache to try to serve assets from the app's earlier crashed state, including after page refreshes. So the end user is still experiencing a crashed app even while Meteor is running following a successful redeployment, which is why I and others thought it was an error in redeployment.

Manually removing the app from the browser manifest (e.g. //appcache-internals in Chrome) fixes things (but of course that's not workable for sites with active users).

@glasser glasser closed this in 062a5a7 Jan 9, 2015

@glasser

This comment has been minimized.

Member

glasser commented Jan 9, 2015

Fixed.

@terenceng2010

This comment has been minimized.

terenceng2010 commented Dec 4, 2015

Our team faces a similar issue when using fast render:

Error: url must be a relative URL: http://51.254.206.142/httptest.php
at [object Object]..extend.classify (packages/routepolicy/routepolicy.js:107:1)
at IsAppUrl (packages/meteorhacks_fast-render/packages/meteorhacks_fast-render.js:138:1)
at [object Object].filterFunction (packages/meteorhacks_fast-render/packages/meteorhacks_fast-render.js:168:1)
at [object Object].PickerImp._dispatch (packages/meteorhacks_picker/packages/meteorhacks_picker.js:44:1)
at processNextSubRouter (packages/meteorhacks_picker/packages/meteorhacks_picker.js:80:1)
at processNextRoute (packages/meteorhacks_picker/packages/meteorhacks_picker.js:73:1)
at processNextMiddleware (packages/meteorhacks_picker/packages/meteorhacks_picker.js:56:1)
....

https://github.com/kadirahq/fast-render/blob/c85e49f08eb0321a0f16ff1fcda8efe80f935e25/lib/server/utils.js
https://github.com/meteor/meteor/blob/dc3cd6eb92f2bdd1bb44000cdd6abd1e5d0285b1/packages/routepolicy/routepolicy.js

As you can see, in fastrender's isAppUrl calls routepolicy.classify, but classify has its own if (url.charAt(0) !== '/') checking that throw the error, it is suitable to modify classify so it calls routepolicy.isValidUrl instead? Or you think there are some changes needed on fast render?

Thanks.

@kaushik1979

This comment has been minimized.

kaushik1979 commented Dec 13, 2015

I updated the meteor version also but I am still getting this error. What am I supposed to do to resolve this error?

@elie222

This comment has been minimized.

Contributor

elie222 commented Mar 1, 2016

I also still see errors like this

@ppryde

This comment has been minimized.

ppryde commented Mar 4, 2016

Yup me too :(

@dovrosenberg

This comment has been minimized.

dovrosenberg commented Jul 19, 2016

Here too.

@hellified

This comment has been minimized.

hellified commented Sep 26, 2016

Got it here too. I'm wondering ... is this related to running the app in a docker container and mapping the port to the host? Is that seen as forwarding by meteor?

@xunga

This comment has been minimized.

xunga commented Feb 18, 2017

Got it as well... I'm so afraid that this might affect SEO.

By the way my configuration:

  • ssr with blaze
  • Service Workers
  • mup
  • https only

errors:
Error: url must be a relative URL: http://www.google.com/
Error: url must be a relative URL: http://www.baidu.com/cache/global/img/gs.gif
Error: url must be a relative URL: http://www.baidu.com/favicon.ico
Error: url must be a relative URL: http://www.bing.com/
Error: url must be a relative URL: http://httpheader.net/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment