Add proxy to support cross-origin requests

[mwbrooks]

Problem:

The PhoneGap Developer App cannot make cross-origin requests because it is served by connect-phonegap.

Source:

phonegap/phonegap-app-developer#98

Solution:

One potential solution is to allow the connect-phonegap middleware to proxy all external requests. An added bonus is that the only the CLI needs to be updated, rather each each app.

[timkim]

Tested and works on:

• iPhone4s - 7.1.1
• Nokia Lumia 920 - Windows Phone 8
• Nexus 5 - Android 4.4

[mwbrooks]

Tested on:

• iPhone 5 7.1.1
• Nokia Lumia 920 WP8
• Galaxy Nexus 4.3

I patched a minor WP8 issue (e6fa4ff) to enable XHRs on local assets.

[slorber]

hey @mwbrooks ,

I've tried the new version of phonegap CLI with PhoneGap App Developer + phonegap serve.

It seems to use the proxy but the url is wrong, maybe because i'm using https scheme into my jquery ajax call?

See the serve logs:

[phonegap] 200 /favicon.ico
Error: Invalid URI "https:/stample.co/authenticateWithoutCookie"
    at Request.init (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/request.js:179:24)
    at new Request (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/request.js:98:8)
    at request (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/index.js:50:11)
    at Object.handle (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/lib/middleware/proxy.js:23:25)
    at next (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.staticMiddleware [as handle] (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/middleware/static.js:55:61)
    at next (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.inject (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect-inject/index.js:106:14)
    at next (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.handle (/usr/lib/node_modules/phonegap/node_modules/connect-phonegap/lib/middleware/autoreload.js:63:13)
[phonegap] 500 /proxy/https:/stample.co/authenticateWithoutCookie

There's a / missing

[mwbrooks]

I'll look into this!

[mwbrooks]

Hi @slorber,

I'm having trouble recreating your issue. I've tested an https: request on Android, iOS, and WP8 devices, but they all work. I've also tried vanilla XHR and jQuery's Ajax call.

Vanilla XHR:

var url = 'https://build.phonegap.com/api/v1/me?auth_token=XXX';
var xhr = new XMLHttpRequest();

xhr.addEventListener('readystatechange', function() {
    alert('state: ' + xhr.readyState + ' | status: ' + xhr.status + ' | responseText: ' + xhr.responseText);
}, false);

xhr.open('get', url, true);
xhr.send();

jQuery 1.11.1:

$.ajax({
    url: 'https://build.phonegap.com/api/v1/me?auth_token=XXX',
    success: function(data, status, xhr) {
        alert('status: ' + status + ' | data: ' + JSON.stringify(data));
    }
});

What device platform are you using? Any chance that you could provide a code sample for me?

[slorber]

Hi @mwbrooks,

I'm using some utility class to perform the request:

var $ = require("jquery");
var Q = require("q");

// Because JQuery promise is sucky (not promise/A+), we use this wrapper
// https://github.com/kriskowal/q/wiki/Coming-from-jQuery

module.exports = function(ajaxOptions) {
    return Q.promise(function (resolve) {
        $.ajax(ajaxOptions)
            .then(function (data, textStatus, jqXHR) {
                delete jqXHR.then; // treat xhr as a non-promise
                resolve(jqXHR);
            }, function (jqXHR, textStatus, errorThrown) {
                delete jqXHR.then; // treat xhr as a non-promise
                resolve(jqXHR);
            });
    });
};

The client code is:

var Qajax = require("common/qajax");
var AppConstants = require("common/appConstants");


// Returns a promise containing the session token
exports.getSessionToken = function(username,password) {
    return Qajax({
        method: "POST",
        url: AppConstants.baseApiUrl + "/authenticateWithoutCookie",
        data: {
            username: username,
            password: password
        }
    })
        .then(function(res) {
            return res.responseJSON.response.session;
        })
};

exports.baseApiUrl = "https://stample.co"

Note that with Ripple / Nexus 4 AVD it works fine.

But with Phonegap App Developer, on my Nexus4, it doesn't work.
I don't know why because I don't know how to debug the problem on the phone, I just see that there's a malformed URL in the proxy logs: Error: Invalid URI "https:/stample.co/authenticateWithoutCookie"

I use Phonegap CLI 3.4.0-0.19.21

I'm available on IRC as Sebastien-L on french business hours to discuss this if needed and can let you access my phonegap serve content

[MrBr]

Have you solved this problem @slorber ?
I have noticed that ajax call goes successful if GET call or POST with NO DATA, if I use ajax POST with data set, then there is this awkward error, long nothing and then Proxy error, ECONNRESET. What I want to say, @mwbrooks try ajax call POST with some data set, here is my example.

<script type="text/javascript">
$(document).bind("ready", function() {
    $.support.cors = true;
    test();
});
</script>

<script>
function test(){
    $.ajax({
        url:'http://www.url.com',
        type:'POST',
        headers:{
            "Content-Type":'application/x-www-form-urlencoded'
        },
        data:'x=1',
        cache:false,
        success: function (data) {
            alert(data);
        },
        error:function(request, textStatus, errorThrown){
            alert(errorThrown);
        }
    });
    return false;
}
</script>

[slorber]

sorry @MrBr , I still have the exact same problem but currently use Ripple, waiting for a fix of @mwbrooks

[MrBr]

Ok, thanks for info. Maybe little of topic, but if you have time to answer @slorber , do you turn off proxy at Ripple or use local/remote ? If I use local, it's not working, remote I don't have any, so I usually disable proxy and everything works.

[slorber]

I start chrome with google-chrome --disable-web-security which permits to bypass the CORS restrictions and be able to make the ajax calls without any proxy, so I can disable it in Ripple

[MrBr]

I see, thanks for info, much appreciated

[slorber]

@mwbrooks here are some debug statements. I have added your JQuery call in my code:

$.ajax({
    url: 'https://build.phonegap.com/api/v1/me?auth_token=XXX',
    success: function(data, status, xhr) {
        alert('status: ' + status + ' | data: ' + JSON.stringify(data));
    }
});

I see the log: [phonegap] 200 /proxy/https:/build.phonegap.com/api/v1/me%3fauth_token=XXX

This still gives me the error.

Maybe this state will help you find the problem?

{
   "domain":null,
   "_maxListeners":10,
   "readable":true,
   "writable":true,
   "method":"GET",
   "__isRequestRequest":true,
   "_events":{

   },
   "uri":{
      "protocol":"https:",
      "slashes":null,
      "auth":null,
      "host":null,
      "port":null,
      "hostname":null,
      "hash":null,
      "search":"?auth_token=XXX",
      "query":"auth_token=XXX",
      "pathname":"/build.phonegap.com/api/v1/me",
      "path":"/build.phonegap.com/api/v1/me?auth_token=XXX",
      "href":"https:/build.phonegap.com/api/v1/me?auth_token=XXX"
   },
   "canTunnel":{
      "httpOverHttp":{

      },
      "httpsOverHttp":{

      },
      "httpOverHttps":{

      },
      "httpsOverHttps":{

      },
      "debug":{

      }
   },
   "pool":{

   },
   "dests":[

   ]
}

It seems the "domain" is not transmitted very well...

I've got exactly the same with JQuery 2.1 or 1.11 and native XHR request like you posted.

I suspect the reason you may not see this error is because you do not use the phonegap serve command but rather use some other server like connect, http or express npm modules (like seen in the doc).

I guess the originally requested url should have been:

/proxy/https://build.phonegap.com/api/v1/me%3fauth_token=XXX

But using 2 slashes in an URL, I don't think it is valid, so it is possible that the http facing layer you are using is normalizing the received url.

[slorber]

I've tried to modify the proxy.js code to check the url before and after at this point:
https://github.com/phonegap/connect-phonegap/blob/master/lib/middleware/proxy.js#L22

It appears that the URI is already decoded and already misses a slash before you try to decode it.
Be aware that I've encountered some servers that automatically decode request params or path params (don't know about node http servers)

[MrBr]

@slorber, I have workaround for testing application at Phonegap developer, I am not sure how does it work with implementing native functions but it serves me pretty well...anyway, instead of listening port 3000, I listen normal 80..you have to run Apache server at local comp, I am using Xampp (wampp)..

So in my case I listen 192.168.1.X/folder-of-app/ + I believe you'll have to add www if using phonegap project structure
X- stand for computer in network, you have to find that number (it is the same number phonegap cli outputs when serve is activated)

Now, as it is not build over nodejs/ phonegap there is no feedback to the application and shortcuts (3/4 finger touch) refresh or go back are not working, so I suggest to add refresh button somewhere at you screen which will refresh code after you change it.

Of course, main thing for me is that ajax cross domain is working like a charm :)

If I haven't explain something best, feel free to ask.

[slorber]

@mwbrooks please also notice that my phone and my phonegap connect servers are not on the same network.

I'm using a service called https://github.com/inconshreveable/ngrok
which permits to expose on the internel a port I have locally opened on my dev computer.

This way, anyone from any network can access my served app from any network with a constant url: http://xxx.ngrok.com

ngrok                                                                                                                                                                         (Ctrl+C to quit)

Tunnel Status                 online                                                                                                                                                          
Version                       1.6/1.6                                                                                                                                                         
Forwarding                    http://xxx.ngrok.com -> 127.0.0.1:8000                                                                                                              
Forwarding                    https://stamplephonegap.ngrok.com -> 127.0.0.1:8000                                                                                                             
Web Interface                 127.0.0.1:4040                                                                                                                                                  
# Conn                        5725                                                                                                                                                            
Avg Conn Time                 198.17ms                                                                                                                                                        



HTTP Requests                                                         
-------------                                                         

POST /proxy/https:/stample.co/                                        
GET /favicon.ico              404 Not Found                           
GET /plugins/org.apache.cordo 200 OK                                  
GET /plugins/org.apache.cordo 200 OK                                  
GET /plugins/org.apache.cordo 200 OK                                  
GET /plugins/org.apache.cordo 200 OK    

I've just tested on the same network, without ngrok and the problem doesn't seem to be here!

The req nodejs object differs if I use ngrok or not:

   "url":"/proxy/http%3A%2F%2Flocalhost%3A9000%2FauthenticateWithoutCookie",

VS

   "url":"/proxy/http:/localhost:9000/authenticateWithoutCookie",

I guess it is an ngrok bug then. When not using ngrok the url logged is correct!

[slorber]

So in the end the url seems correct, but now I have another bug: the POST request is never transmitted to my local server.

Short version:

POST with form data of type application/x-www-form-urlencoded doesn't seem to work with the server proxy, probably because of a bug in request framework, or maybe the proxying of form data does not work with simple request to proxy piping.

Here you can find a jsfiddle that use the local proxy launched with phonegap serve --port 8000

http://jsfiddle.net/3wM8c/

The POST request keeps running forever (when it has data only!)

Long version:

The JS request on the mobile phone is:

$.ajax({
        method: "POST",
        url: "http://localhost:9000" + "/authenticateWithoutCookie",
        data: {
            username: "xxx",
            password: "yyy"
        }
    })

The request received by the proxy is: https://gist.github.com/slorber/7902a6ab0bd38aae53f7

The request forwarding doesn't seem to be fired at all.
I've tried various local modifications of your proxy code like

// Permits to log the request without circular ref problems
var cache=[];
function stringifyCache(key, value) {
    if (typeof value === 'object' && value !== null) {
        if (cache.indexOf(value) !== -1) {
            // Circular reference found, discard key
            return;
        }
        // Store value in our collection
        cache.push(value);
    }
    return value;
}




module.exports = function(options) {
    return function(req, res, next) {

    function log(string) {
            options.emitter.emit('log', string);
    }


        if (req.url.indexOf('/__api__/proxy/') === 0 || req.url.indexOf('/proxy/') === 0) {
            var url = decodeURIComponent(req.url.replace('/proxy/', ''));

            log('Will proxy test: ' + url);
            log('Will proxy req: ' + JSON.stringify(req,stringifyCache));

        req.pipe(request(url)).pipe(fs.createWriteStream('/home/sebastien/Desktop/test'));
        request(url).pipe(fs.createWriteStream('/home/sebastien/Desktop/test2'));

            var proxy = request(url);
            proxy.on('data', function(data) { log('Proxy data: url: ' + url + "\n" + data); });
            proxy.on('error', function(error) { log('Proxy error: url: ' + url + "\n" + error); });
            proxy.on('end', function() { log('Proxy end: url: ' + url); });

            req.pipe(proxy);
            proxy.pipe(res);

            res.on('data', function(data) { log('Res data: url: ' + url + "\n" + data); });
            res.on('error', function(error) { log('Res error: url: ' + url + "\n" + error); });
            res.on('end', function() { log('Res end: url: ' + url); });
        }
        else {
            next();
        }
    };
};

On my mobile phone, the request seems to keep being running forever.

The only logs I see are:

[phonegap] Will proxy test: http://localhost:9000/authenticateWithoutCookie undefined
[phonegap] Will proxy req: (see gist above)

None of the on data/error/end callbacks are fired. A file is created at /home/sebastien/Desktop/test but it contains nothing.

The file at /home/sebastien/Desktop/test2 is written correctly but as it is a GET request while the service expects a POST, the result is actually an html error page.

So it seems the following doesn't work so well:

            req.pipe(proxy);
            proxy.pipe(res);

Because no request gets fired.

To demonstrate the problem. I tried to test the proxy manually with curl and a remote service called http://requestb.in/ which permits to receive http requests and log them for introspection.

sebastien@sebastien-xps:stample-Phonegap (master *%)$ curl -X GET http://localhost:8000/proxy/http%3A%2F%2Frequestb.in%2F17lj7rm1
ok
sebastien@sebastien-xps:stample-Phonegap (master *%)$ curl -X POST http://localhost:8000/proxy/http%3A%2F%2Frequestb.in%2F17lj7rm1%3Fusername%3Dxxx%26password%3Dyyy
ok
sebastien@sebastien-xps:stample-Phonegap (master *%)$ curl -X POST http://localhost:8000/proxy/http%3A%2F%2Frequestb.in%2F17lj7rm1 --data "username=xxx&password=yyy"

The last call keeps being pending and no request is logged on http://requestb.in/

I guess this is a bug, maybe from https://github.com/mikeal/request ?

@mwbrooks can you please try to use this code in your test, a POST with some data?

$.ajax({
        method: "POST",
        url: "anyLoginUrl",
        data: {
            username: "xxx",
            password: "yyy"
        }
    })

[mwbrooks]

@slorber would you mind updating to the latest CLI and giving it another shot? We've added a patch from #75. I'll leave this open until we get some confirmation that it's helped.

[slorber]

Sorry @mwbrooks but it doens't seem to solve my problem.

[phonegap] 200 /__api__/autoreload
Error: Invalid URI "https:/stample.co/authenticateWithoutCookie"
    at Request.init (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/request.js:179:24)
    at new Request (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/request.js:98:8)
    at request (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/request/index.js:50:11)
    at Object.handle (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/lib/middleware/proxy.js:39:25)
    at next (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.staticMiddleware [as handle] (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/middleware/static.js:55:61)
    at next (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.inject (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect-inject/index.js:106:14)
    at next (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/node_modules/connect/lib/proto.js:193:15)
    at Object.handle (/home/sebastien/Desktop/devhome/install/node-v0.10.24-linux-x64/lib/node_modules/phonegap/node_modules/connect-phonegap/lib/middleware/autoreload.js:78:13)
[phonegap] 500 /proxy/https:/stample.co/authenticateWithoutCookie

sebastien@sebastien-xps:stample-Phonegap (master $)$ phonegap --version
3.5.0-0.20.7

[MrBr]

Before there was no error (for me), just long nothing until I stopped CLI and then some Proxy error in one line. Was your case same before ?

You have invalid URI, "https:/stample.co/authenticateWithoutCookie" -> https://
As I can see you miss one slash.

[slorber]

@MrBr yes this is exactly the point! the problem is that it is not me that only put a single slash!

@mwbrooks according to my tests this is much better, since I'm able to make my request through my local proxy using both CURL and this JSFiddle http://jsfiddle.net/3wM8c/1/ while it didn't work before.

So, for me the problem with connect-phonegap has been fixed, as the proxy seems to work fine.

However I am still unable to use the proxy on my mobile with the Phonegap App Developer :( I guess this time it is the mobile app that removes a slash.

Error: Invalid URI "https:/stample.co/authenticateWithoutCookie"

Notice that this app works perfectly fine with Ripple, AVD, deployed as dev APK. It is even published on Google Play Store! https://play.google.com/store/apps/details?id=co.stample&hl=en
The only this that does not work is Phonegap App Developer for me

[MrBr]

@slorber I understand, unfortunately, don't have any clue.
Have you updated your mobile app ? best I could come with.

I have done few more test and noticed that if data was sent in object type, request wasn't made again, so did fix again, we'll have to wait for @mwbrooks to check it.

[slorber]

I have the mobile app version 1.1.0 and it seems I can't upgrade it through the store so I guess it's the last one

[mwbrooks]

Closing. Support was added a while back.