New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HTTP package] Pass unrecognized options through to request module #1703

Closed
dandv opened this Issue Dec 22, 2013 · 17 comments

Comments

Projects
None yet
10 participants
@dandv
Contributor

dandv commented Dec 22, 2013

The HTTP package calls request, which in turn calls Node's http or https modules. The latter especially can take rather esoteric options for TSL configuration.

It would be great if HTTP.call() would pass through any options it doesn't recognize. This solves a use case I ran into, where I had to set {rejectUnauthorized: false}.

@glasser

This comment has been minimized.

Member

glasser commented Jan 8, 2014

The goal of the HTTP package is to provide a consistent interface between client and server, so adding lots of server-only options doesn't seem look a good idea. Maybe a specific option that's called "extraNodeRequestOptions" and documented as "We might change the backend to not be request and this option will break at that point."

That said, if you want to do complex HTTP things that the Meteor http package doesn't implement, there's the workaround of just using Npm.require('http') or Npm.require('request') (the latter with Npm.depends in a package, though it may coincidentally work for now without that since request is in the dev bundle) yourself.

@glasser glasser closed this Jan 8, 2014

@dandv

This comment has been minimized.

Contributor

dandv commented Feb 27, 2014

I and others who've requested proxy support on SO would be happy to have an extraNodeRequestOptions option.

As for a consistent interface between client and server, the problem is similar to Minimongo vs. MongoDB. The latter has more options than the former, which is fantastic. Right now the client can't perform CORS requests for instance, while that's not a problem for the server.

The client and server are just different.

Why not empower the server to use proxies? Presumably the feature would be kept in the event that meteor would switch away from the request backend.

@raix

This comment has been minimized.

Contributor

raix commented Feb 27, 2014

@dandv agree #1814
If the http should be consistent between client and server well shouldn't followredirects go then?

@dandv

This comment has been minimized.

Contributor

dandv commented Mar 25, 2014

For those who don't want to reinvent the wheel and use the low-level Npm https module, I've created a fork of Meteor's HTTP package called http-more.

@guaka

This comment has been minimized.

guaka commented Jul 4, 2014

I'd like to have the possibility to use a proxy, especially if it's somehow possible to let the client side revert to proxy chaining, (http://proxy.example.com/http://server.example.com, see also http://stackoverflow.com/questions/6988441/can-client-javascript-use-its-own-http-proxy)

@LeCoupa

This comment has been minimized.

LeCoupa commented Sep 24, 2014

The package http-more is now available here. (Dan's url seems to redirect to a previous version of the package)

@ghost

This comment has been minimized.

ghost commented Sep 29, 2014

Surely there must be a way to specify an outbound proxy server for meteor to use ? Meteor is useless in a corporate environment without that. Client side is handled automatically by the browser config so is transparent, but server has to explicitly configured somewhere. Industry standard http_proxy environment variable would be nice.

@mitar

This comment has been minimized.

Collaborator

mitar commented Dec 2, 2014

I cannot believe that after more than a year after my #1664, this is still not fixed. Really crazy. It is really so hard to add some underscore prefixed options while we are waiting for "perfect" solution?

@dandv

This comment has been minimized.

Contributor

dandv commented Dec 2, 2014

@LeCoupa: http-more is available at https://github.com/dandv/meteor-http-more, can you please update the link in your comment or delete it (I've fixed mine above)? Thanks.

I agree with @mitar and I'll be happy to retire my package in favor of Meteor's HTTP package simply passing along options it doesn't specifically recognize.

@mitar

This comment has been minimized.

Collaborator

mitar commented Dec 2, 2014

In fact I need https://github.com/aldeed/meteor-http-extras. :-) I need to specify binary options.

@aldeed

This comment has been minimized.

Contributor

aldeed commented Dec 2, 2014

@mitar, I am biased but I have to agree. I put effort into making sure my PR (which is now the aldeed:http add-on package) fixed the glaring problems with the current package while keeping the client and server API the same, other than a few tiny differences that are unavoidable.

I think the issue is probably that MDG just doesn't see any need for an official http package anymore, so it won't be high priority. Our best bet is probably to combine all the features of http plus aldeed:http plus http-more into a new community-supported package that is a complete replacement for http. It's really not even a huge undertaking, but I don't have time in the foreseeable future.

@niallobrien

This comment has been minimized.

niallobrien commented Dec 17, 2014

The real world shows us that client apps are appearing on devices that were never considered when Meteor was first created, eg. watches, microwaves, cars, refrigerators etc. We cannot be expected to use Meteor across all of these. More server-side options to get at our data is greatly needed.
Furthermore, Meteor was clearly never designed to scale in a cost-efficient manner. Having each DDP client eat RAM on the server is silly. There are plenty of use cases for alternative methods of getting data in and out of our Meteor apps, without always having to buy into the entire ecosystem.

@Anonyfox

This comment has been minimized.

Anonyfox commented Dec 21, 2014

As someone who has to deal with mostly non-english content, it outright sucks that i just can't set {encoding: null} to get a buffer instead of an utf-8 string. Really. I have to deal with crappy (and wrong encoded) websites using iconv-lite and just to set {encoding: null} I have to jump through some hoops with third party packages.

@glasser

This comment has been minimized.

Member

glasser commented Feb 17, 2015

Feedback has been heard. We should make it much easier to use request directly via the HTTP module, with appropriate warnings that the API could change with updates. I hope to do this for 1.0.4 if time permits.

@glasser glasser reopened this Feb 17, 2015

glasser added a commit that referenced this issue Mar 4, 2015

http: Allow specifying npm request options
You can use HTTP.NpmModules.request.version to tell what version of
request (if any) is the backend for HTTP.call.  This version may change
incompatibly from version to version of Meteor; use at your own risk.

Fixes #1703.

glasser added a commit that referenced this issue Mar 5, 2015

http: Allow specifying npm request options
You can use HTTPInternals.NpmModules.request.version to tell what
version of request (if any) is the backend for HTTP.call.  This version
may change incompatibly from version to version of Meteor; use at your
own risk.

Fixes #1703.

glasser added a commit that referenced this issue Mar 5, 2015

http: Allow specifying npm request options
You can use HTTPInternals.NpmModules.request.version to tell what
version of request (if any) is the backend for HTTP.call.  This version
may change incompatibly from version to version of Meteor; use at your
own risk.

Fixes #1703.

@glasser glasser closed this in 8dae78d Mar 6, 2015

@dandv

This comment has been minimized.

Contributor

dandv commented Jul 30, 2015

@glasser, thanks for reconsidering this feature request and implementing a fix.

For others who check this issue - you can now pass npmRequestOptions to HTTP.call.

@digz6666

This comment has been minimized.

digz6666 commented Jan 4, 2016

Any example using npmRequestOptions?

@digz6666

This comment has been minimized.

digz6666 commented Jan 4, 2016

Nevermind, I found it:

HTTP.post(url, {
    npmRequestOptions: {
        rejectUnauthorized: false // TODO remove when deploy
    },
    timeout: 30000, // 30s
    data: xml
}, function(error, result) {
    console.log('error: ' + error);
    console.log('resultXml: ' + result);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment