HTTP Requests generate different payload than requests using sam local invoke

[dewski]

I'm in the process of developing my first couple Lambda functions and doing various testing locally and after deploying to Prod.

I've been having trouble interacting with my functions locally or in Prod using HTTP. All the documentation points to being able to interact with your function using HTTP.

After running $ sam local start-api I am presented with (which mentions you can invoke it locally):

2019-01-16 12:21:09 Found credentials in shared credentials file: ~/.aws/credentials
2019-01-16 12:21:09 Mounting ComponentLabeling at http://127.0.0.1:3000/component-labeling [POST]
2019-01-16 12:21:09 You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2019-01-16 12:21:09  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

When invoking the endpoint using the payload as described by the documentation:

curl -XPOST -H"Content-Type: application/json" "http://localhost:3000/component-labeling" -d '{
  "image_url": "urltopng"
}'

I am presented with:

Fetching lambci/lambda:go1.x Docker container image......
2019-01-16 12:22:20 Mounting /Users/dewski/go/src/github.com/dewski/pngdiff/component-labeling as /var/task:ro inside runtime container
START RequestId: 38f38f66-717c-1a28-a078-c21bc737a725 Version: $LATEST
END RequestId: 38f38f66-717c-1a28-a078-c21bc737a725
REPORT RequestId: 38f38f66-717c-1a28-a078-c21bc737a725	Duration: 1.69 ms	Billed Duration: 100 ms	Memory Size: 3008 MB	Max Memory Used: 6 MB	
{
  "errorMessage": "missing valid image_url got \"\"",
  "errorType": "errorString"
}

My go application looks like this:

package main

import (
	"errors"
	"fmt"
	"net/url"

	"github.com/aws/aws-lambda-go/lambda"
	"github.com/dewski/pngdiff/cmd/pngdiff"
)

func validURL(input string) bool {
	if input == "" {
		return false
	}

	_, err := url.Parse(input)
	return err == nil
}

// Payload handles the incoming request.
type Payload struct {
	ImageURL          string `json:"image_url"`
	MinimumRegionArea int    `json:"minimum_region_area"`
}

// Response handles the response.
type Response struct {
	Regions []*pngdiff.Region `json:"regions"`
}

// ComponentLabeling finds components in an image and labels them.
func ComponentLabeling(req Payload) (Response, error) {
	fmt.Println(req)
	if !validURL(req.ImageURL) {
		return Response{}, fmt.Errorf("missing valid image_url got \"%s\"", req.ImageURL)
	}

	image, err := pngdiff.DownloadImage(req.ImageURL)
	if err != nil {
		return Response{}, errors.New("could not download image_url")
	}

	regions, err := pngdiff.DetectRegions(image)
	if err != nil {
		return Response{}, err
	}

	filteredRegions := []*pngdiff.Region{}
	for _, r := range regions {
		if r.Area() < req.MinimumRegionArea {
			continue
		}

		filteredRegions = append(filteredRegions, r)
	}

	return Response{
		Regions: filteredRegions,
	}, nil
}

func main() {
	lambda.Start(ComponentLabeling)
}

It matches the same handler signature as outlined in the documentation and in lambda/entry.go:

aws-lambda-go/lambda/entry.go

Line 31 in deb5700

// func (TIn) (TOut, error)

When I invoke my function using sam local invoke, everything works as expected:

echo '{
  "image_url": "http://sitegif-development.s3.amazonaws.com/4f01e5e97d7d0aba4ee6959ea521e5c3/78d48838-f669-4563-a866-45639211d78c-633ea5d6-b662-4e62-956a-2cdcfe3b8030-comparison.png"
}' | sam local invoke "ComponentLabeling"

2019-01-16 12:25:45 Reading invoke payload from stdin (you can also pass it from file with --event)
2019-01-16 12:25:45 Found credentials in shared credentials file: ~/.aws/credentials
2019-01-16 12:25:45 Invoking component-labeling (go1.x)

Fetching lambci/lambda:go1.x Docker container image......
2019-01-16 12:25:46 Mounting /Users/dewski/go/src/github.com/dewski/pngdiff/component-labeling as /var/task:ro inside runtime container
{"regions":[{

(output truncated for brevity)

Really puzzled, I did some digging and noticed the handler function will be invoked with two different payloads depending on how the request was triggered (HTTP or sam):

aws-lambda-go/lambda/function.go

Lines 59 to 65 in deb5700

	payload, err := fn.handler.Invoke(invokeContext, req.Payload)
	if err != nil {
	response.Error = lambdaErrorResponse(err)
	return nil
	}
	response.Payload = payload
	return nil

When I log the output of req.Payload using HTTP I get:

{
  "httpMethod": "POST",
  "body": "{\n  \"image_url\": \"http:\/\/sitegif-development.s3.amazonaws.com\/4f01e5e97d7d0aba4ee6959ea521e5c3\/78d48838-f669-4563-a866-45639211d78c-633ea5d6-b662-4e62-956a-2cdcfe3b8030-comparison.png\"\n}",
  "resource": "\/component-labeling",
  "requestContext": {
    "resourceId": "123456",
    "apiId": "1234567890",
    "resourcePath": "\/component-labeling",
    "httpMethod": "POST",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "accountId": "123456789012",
    "stage": "prod",
    "identity": {
      "apiKey": null,
      "userArn": null,
      "cognitoAuthenticationType": null,
      "caller": null,
      "userAgent": "Custom User Agent String",
      "user": null,
      "cognitoIdentityPoolId": null,
      "cognitoAuthenticationProvider": null,
      "sourceIp": "127.0.0.1",
      "accountId": null
    },
    "extendedRequestId": null,
    "path": "\/component-labeling"
  },
  "queryStringParameters": null,
  "headers": {
    "Host": "localhost:3000",
    "User-Agent": "curl\/7.54.0",
    "Accept": "*\/*",
    "Content-Length": "186",
    "Content-Type": "application\/x-www-form-urlencoded",
    "X-Forwarded-Proto": "http",
    "X-Forwarded-Port": "3000"
  },
  "pathParameters": null,
  "stageVariables": null,
  "path": "\/component-labeling",
  "isBase64Encoded": false
}

When invoking the function using sam the payload is the expected payload:

{
  "image_url": "http://sitegif-development.s3.amazonaws.com/4f01e5e97d7d0aba4ee6959ea521e5c3/78d48838-f669-4563-a866-45639211d78c-633ea5d6-b662-4e62-956a-2cdcfe3b8030-comparison.png"
}

Is this a bug or expected behavior? If it's expected behavior, how does one successfully invoke a function using HTTP?

[bmoffatt]

sam local start-api mimics the api gateway proxy integration, which is what produces the envelope json you observed. When using Lambda with api gateway, sam local start-api is the right way to test locally. Also when using api gateway, the function should accept events.APIGatewayProxyRequest and return events.APIGatewayProxyResponse (example)