Serverless Deploy behind corporate proxy

[felixbecker]

Hi, I try to deploy the standard hello world appliction behind a corporate proxy from a windows client.
I tryed it also on a linux vm an it also didn't work.
Trying the same thing with the same computer with a direct connection it worked.

Description

For bug reports:

• What went wrong?
  Deployment behind a corporate proxy does not work.

• What did you expect should have happened?
  serverless deploy should deploy hello world function

• What was the config you used?

service: hello-world
provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  region: eu-central-1

functions:
  hello:
    handler: handler.hello

Standard hello world handler code

'use strict';


module.exports.hello = (event, context, callback) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: 'Go Serverless v1.0! Your function executed successfully!',
      input: event,
    }),
  };

  callback(null, response);

  // Use this code if you don't use the http event with the LAMBDA-PROXY integration
  // callback(null, { message: 'Go Serverless v1.0! Your function executed successfully!', event });
};

• What stacktrace or error message from your provider did you see?

  Serverless Error ---------------------------------------

     Unexpected close tag
Line: 115
Column: 7
Char: >

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                 win32
     Node Version:       6.10.2
     Serverless Version: 1.11.0

Stacktrace with SLS_DEBUG=*

ServerlessError: Unexpected close tag Line: 115 Column: 7 Char: > at Response.req.send (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:163:20) at Request.<anonymous> (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:355:18) at Request.callListeners (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\sequential_executor.js:105:20) at Request.emit (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\sequential_executor.js:77:10) at Request.emit (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:668:14) at Request.transition (D:\Users\MyUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:22:10) at AcceptorStateMachine.runTo (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\state_machine.js:14:12) at D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\state_machine.js:26:10 at Request.<anonymous> (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:38:9) at Request.<anonymous> (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:670:12) at Request.callListeners (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\sequential_executor.js:115:18) at Request.emit (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\sequential_executor.js:77:10) at Request.emit (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:668:14) at Request.transition (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\request.js:22:10) at AcceptorStateMachine.runTo (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\state_machine.js:14:12) at D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\node_modules\aws-sdk\lib\state_machine.js:26:10 From previous event: at persistentRequest (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:151:14) at doCall (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:130:9) at BbPromise (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:141:14) From previous event: at persistentRequest (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:128:38) at AwsProvider.request (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\provider\awsProvider.js:144:12) at AwsDeploy.BbPromise.bind.then.then (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\deploy\lib\configureStack.js:23:35) From previous event: at AwsDeploy.configureStack (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\deploy\lib\configureStack.js:23:10) From previous event: at Object.deploy:initialize [as fn] (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\plugins\aws\deploy\index.js:42:10) at BbPromise.reduce (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:210:55) at runCallback (timers.js:666:20) at tryOnImmediate (timers.js:639:5) at processImmediate [as _immediateCallback] (timers.js:611:5) From previous event: at PluginManager.invoke (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:210:22) at PluginManager.run (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\classes\PluginManager.js:225:17) at Serverless.run (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\lib\Serverless.js:97:31) at serverless.init.then (D:\Users\myUser\AppData\Roaming\npm\node_modules\serverless\bin\serverless:23:50)

Unfortunately I did not find anything while googleling for that error and the proxy setting HTTP_PROXY and HTTPS_PROXY are set.
Is there anything I am missing here?
Any help would be greate. Thx

[pmuens]

That's interesting. Seems like the request to AWS is not processed correctly.

🤔 is it possible for you to use the aws-cli in your terminal?

Here's where the error is thrown: https://github.com/serverless/serverless/blob/master/lib/plugins/aws/provider/awsProvider.js#L163

Just pasting this link to the issues (filtered by "company proxy") of the AWS SDK JS GitHub repo here (which might help): https://github.com/aws/aws-sdk-js/issues?utf8=%E2%9C%93&q=is%3Aissue%20company%20proxy%20is%3Aclosed%20

[felixbecker]

Hi thanks for the quick reply.

When working with aws-sdk straight I usually do something like this:

var AWS = require('aws-sdk');
var proxy = require('proxy-agent');

AWS.config.update({
  httpOptions: { 
    agent: proxy('http://user:password@internal.proxy.com') 
  }
});

like descibed here:
https://aws.amazon.com/de/blogs/developer/using-the-aws-sdk-for-javascript-from-behind-a-proxy/

aws-cli does also work: aws s3 ls shows my list

the links from aws-sdk-js do not apply. These are the common errors when proxy is not defined. After setting HTTP_PROXY and HTTPS_PROXY I got my reported Error above.

[felixbecker]

The source code file provided a good hint.

Acutally this line helped: https://github.com/serverless/serverless/blob/master/lib/plugins/aws/provider/awsProvider.js#L132

Interestingly I got an http error 407 (Proxy Authentication). Since this is piped through all the way the parser tries to parse the response. But the response contains a html page from the proxy. This is what causes the parsing error. So it looks like this is not an serverless issue but it would be nice to handle that error with a message.

basically something like
if (e.statusCode === 407){ ... // Proxy authentication required // .... }
could help .. if it would align with your api strategie =)...

[pmuens]

That makes sense. Thanks for looking into it @felixbecker 👍

I'll keep this issue open and label it with the easy-pick label. Maybe someone is kind enough to PR a fix for that.

[jason-riddle]

Two questions.

1. So for the proxy 407 error, maybe have something like this?

 const persistentRequest = (f) => new BbPromise((resolve, reject) => {
   const doCall = () => {
     f()
       .then(resolve)
       .catch((e) => {
         if (e.statusCode === 429) {
           that.serverless.cli.log("'Too many requests' received, sleeping 5 seconds");
           setTimeout(doCall, 5000);
+        } else if (e.statusCode === 407) {
+          that.serverless.cli.log("'Proxy Authentication' received, exiting");
+          reject(e);
+        } else {
           reject(e);
         }
       });
   };
   return doCall();
 });

2. For the proxy definition in Serverless Deploy behind corporate proxy #3508 (comment), where does that change need to be made?

[pmuens]

@jason-riddle thanks for chiming in 👍

The code looks good. Maybe we can add some more info or a hint why Serverless will exit. So that the user knows what's going on and how he can resolve it (we could e.g. add a link to some docs here).

What do you think about that?

2. For the proxy definition in Serverless Deploy behind corporate proxy #3508 (comment), where does that change need to be made?

Pinging @felixbecker for some feedback on that 🔝

[felixbecker]

@pmuens and @jason-riddle , sorry for the late reply I was kind of busy the last days. And thanks for jumping in. The code for the fix looks good 👍

According to the #3508 proxy definition:
As far as I could see, the code does contain already something similar for the proxy definition in the following lines.
https://github.com/serverless/serverless/blob/master/lib/plugins/aws/provider/awsProvider.js#L105-L117

I think I need to verify a little deeper if this is just a settings issue on our site or if we have a n issue.

Thanks again for the quick reply

I think there are two environment variables when the proxy is set: http_proxy and https_proxy. If so, can it help?

[pmuens]

@diegotry thanks for the pointers 👍

Seems like we're already using those here:

serverless/lib/plugins/aws/provider/awsProvider.js

Lines 106 to 110 in cc941f8

	const proxy = process.env.proxy
	|| process.env.HTTP_PROXY
	|| process.env.http_proxy
	|| process.env.HTTPS_PROXY
	|| process.env.https_proxy;

🤔

[felixbecker]

Hi @diegotry, @pmuens thx everyone, sure I know that proxy support is implemented but it some how does not work.

even when I set HTTP[S]_PROXY=http://{username}:{password}@proxy:port

I got the 407 Authentication error. I am not sure if this is a configuration thing on our site ... but I somehow do not find time to test that extensivly.
My proxy settings in npm work just fine. I found also some similar messages in the aws-sdk.js repo.
Sorry that I cannot be more precise but it just does not work in the coperate enviroment (but like I said this can also come from the IT lock down).

I'll keep you posted.
p.s: If you prefer to close the issues I'll be fine and happy to return if I have any results here.

[pmuens]

Sorry that I cannot be more precise but it just does not work in the coperate enviroment (but like I said this can also come from the IT lock dow

Thanks for taking the time to look into it. Might be really something weird going on with the SDK.

p.s: If you prefer to close the issues I'll be fine and happy to return if I have any results here.

That sounds like a good idea! Just keep us posted and we'll repone!

Thanks everyone for the help so far 👍 !