Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setting up debugger with VSCode with Golang Issue #1067

Closed
petrogko opened this issue Mar 19, 2019 · 11 comments
Closed

Setting up debugger with VSCode with Golang Issue #1067

petrogko opened this issue Mar 19, 2019 · 11 comments

Comments

@petrogko
Copy link

petrogko commented Mar 19, 2019 •
edited
Loading

Description

Permission issue while attempting to execute a curl request on a built binary that should allow me to debug aws lambdas locally.

Steps to reproduce

  1. GOOS=linux GOARCH=amd64 go build -gcflags='-N -l' -o bin/main invite/main.go
  2. GOOS=linux GOARCH=amd64 go build -o bin/dlv ~/go/src/github.com/go-delve/delve/cmd/dlv
  3. sam local start-api -d 5858 --debugger-path ./bin --debug-args "-delveAPI=2"

template.yaml
`AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
invite-bp

Sample SAM Template for invite-bp

More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst

Globals:
Function:
Timeout: 5

Resources:
InviteGetFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: ./bin
Handler: main
Runtime: go1.x
Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html
Events:
CatchAll:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /invite
Method: ANY
Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
Variables:
PARAM1: VALUE

InvitePostFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
#Policies:
# - DynamoDBCrudPolicy:
# TableName: !Ref FeelingsPollResults
CodeUri: ./bin
Handler: main
Runtime: go1.x
Tracing: Active # https://docs.aws.amazon.com/lambda/latest/dg/lambda-x-ray.html
Events:
PutRequest:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /invite
Method: POST
Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
Variables:
PARAM1: VALUE

Outputs:

ServerlessRestApi is an implicit API created out of Events key under Serverless::Function

Find out more about other implicit resources you can reference within SAM

https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api

InviteAPI:
Description: "API Gateway endpoint URL for Prod environment for First Function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/invite/"
InviteGetFunction:
Description: "Fetching Invite"
Value: !GetAtt InviteGetFunction.Arn
InviteGetFunctionIamRole:
Description: "Implicit IAM Role created for Invite function"
Value: !GetAtt InviteGetFunctionRole.Arn
InvitePostFunction:
Description: "Creating Invite"
Value: !GetAtt InvitePostFunction.Arn
InvitePostFunctionIamRole:
Description: "Implicit IAM Role created for Invite function"
Value: !GetAtt InvitePostFunctionRole.Arn
`

Observed result

Fetching lambci/lambda:go1.x Docker container image......... 2019-03-18 21:28:09 Mounting /Users/x/go/src/x/x/invite-bp/bin as /var/task:ro inside runtime container Could not create config directory: mkdir /home/sbx_user1051: permission denied.API server listening at: [::]:5858 time="2019-03-19T01:28:10Z" level=info msg="launching process with args: [/var/task/invite]" layer=debugger could not launch process: fork/exec /var/task/invite: no such file or directory

Expected result

Run through vscode debugger step by step in order to determine issues with code.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Mac Mojave 10.14.3
  2. sam --version: SAM CLI, version 0.13.0
@petrogko petrogko mentioned this issue Mar 19, 2019
Closed
@jackmcguire1
Copy link

ok i don't understand now because your template is using main as the handler yet your error is '/var/task/invite' which means this is an old log

@petrogko
Copy link
Author

petrogko commented Mar 19, 2019 •
edited
Loading

@jackmcguire1 - I apologize, I have to many bash windows open. Copied it from wrong one.
Just re-ran this with suggestions from previous post. Issue remains the same.

Fetching lambci/lambda:go1.x Docker container image......... 2019-03-19 12:45:59 Mounting /Users/x/go/src/x/invite-bp/bin as /var/task:ro inside runtime container Could not create config directory: mkdir /home/sbx_user1051: permission denied.API server listening at: [::]:5858 time="2019-03-19T16:46:00Z" level=info msg="launching process with args: [/var/task/main]" layer=debugger could not launch process: fork/exec /var/task/main: no such file or directory

@jackmcguire1
Copy link

jackmcguire1 commented Mar 19, 2019 •
edited
Loading

i think best case scenario is to start a-fresh and build on that, your current template is full of comments and junk and getting me confused.

`AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31

Globals:
Function:
Runtime: go1.x
Timeout: 300
Tracing: Active
Handler: main

Resources:
MyApi:
Type: AWS::Serverless::Api
Properties:
StageName: Prod

Invite:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./dist/invite/
Events:
GetInvite:
Type: Api
Properties:
RestApiId: !Ref MyApi
Path: "/invite/{id}"
Method: Get
FunctionName: !Sub ${AWS::StackName}-Invite

Outputs:
ApiURL:
Description: "API URL"
Value: !Sub "https://${MyApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
`

GOARCH=amd64 GOOS=linux go build -o ./dlv github.com/go-delve/delve/cmd/dlv

GOARCH=amd64 GOOS=linux go build -gcflags='-N -l' -o dist/invite/main ./invite/main.go

sam local start-api --debugger-path . --debug-args="-delveAPI=2" --log-file ./output.log

@petrogko petrogko reopened this Mar 19, 2019
@petrogko
Copy link
Author

Sorry, accidentally closed the issue.
Issue still occurs.

@jackmcguire1 - I have used your template to re-create this. Unfortunately same problem.

@jfuss
Copy link
Contributor

jfuss commented Mar 21, 2019

@petrogdev I am not sure what you are exactly doing but I am able to debug Go in VSCode. Here were my steps:

sam init --runtime go1.x
cd sam-app
code . # to launch VSCode
go get -u github.com/go-delve/delve/cmd/dlv
GOARCH=amd64 GOOS=linux go build -o ./dlv github.com/go-delve/delve/cmd/dlv
go get -u github.com/aws/aws-lambda-go/...
GOARCH=amd64 GOOS=linux go build -gcflags='-N -l' -o hello-world/hello-world ./hello-world
sam local start-api -d 5986 --debugger-path . --debug-args="-delveAPI=2"

In VSCode my launch config is the follow (copied from our docs and added the debug port specified in the command).

{
    "version": "0.2.0",
    "configurations": [
    {
        "name": "Connect to Lambda container",
        "type": "go",
        "request": "launch",
        "mode": "remote",
        "remotePath": "",
        "port": "5986",
        "host": "127.0.0.1",
        "program": "${workspaceRoot}",
        "env": {},
        "args": [],
      },
    ]
}

I hit http://localhost:3000/hello from my browser and then set a breakpoint in VSCode and launched the debugger. Stepped through some lines. Here is the output from the command line:

(aws-sam-cli) ➜  sam-app sam local start-api -d 5986 --debugger-path . --debug-args="-delveAPI=2"
2019-03-21 10:04:06 Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
2019-03-21 10:04:06 You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2019-03-21 10:04:06  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
2019-03-21 10:07:01 Invoking hello-world (go1.x)

Fetching lambci/lambda:go1.x Docker container image......................................................
2019-03-21 10:07:07 Mounting /Users/jfuss/sam-playground/sam-app/hello-world as /var/task:ro inside runtime container
Could not create config directory: mkdir .config: read-only file system.API server listening at: [::]:5986
time="2019-03-21T17:07:08Z" level=info msg="launching process with args: [/var/task/hello-world]" layer=debugger
time="2019-03-21T17:08:51Z" level=info msg="created breakpoint: &api.Breakpoint{ID:1, Name:\"\", Addr:0x6e596e, File:\"/Users/jfuss/sam-playground/sam-app/hello-world/main.go\", Line:25, FunctionName:\"main.handler\", Cond:\"\", Tracepoint:false, TraceReturn:false, Goroutine:false, Stacktrace:0, Variables:[]string(nil), LoadArgs:(*api.LoadConfig)(0xc000c61740), LoadLocals:(*api.LoadConfig)(0xc000c61770), HitCount:map[string]uint64{}, TotalHitCount:0x0}" layer=debugger
time="2019-03-21T17:08:51Z" level=debug msg=continuing layer=debugger
START RequestId: 27101d69-3934-1c3e-f5f1-a35f6db2a2dd Version: $LATEST
time="2019-03-21T17:08:56Z" level=debug msg=nexting layer=debugger
time="2019-03-21T17:09:00Z" level=debug msg=continuing layer=debugger
END RequestId: 27101d69-3934-1c3e-f5f1-a35f6db2a2dd
REPORT RequestId: 27101d69-3934-1c3e-f5f1-a35f6db2a2dd	Duration: 9308.64 ms	Billed Duration: 9400 ms	Memory Size: 128 MB	Max Memory Used: 76 MB
2019-03-21 10:09:01 No Content-Type given. Defaulting to 'application/json'.
2019-03-21 10:09:01 127.0.0.1 - - [21/Mar/2019 10:09:01] "GET /hello HTTP/1.1" 200 -
2019-03-21 10:09:01 127.0.0.1 - - [21/Mar/2019 10:09:01] "GET /favicon.ico HTTP/1.1" 403 -

You are seeing a file note found on your binary. Which could mean a couple things,

  • Docker does not have access to share files from the host to the container. You can go into docker's settings and make sure the drive or directory of where you are working is shared. If you are on Windows, you may need to toggle and restart (Not sure why but sometimes Docker on Windows needs a reset).
  • You are building the function's binary in a place outside the specified CodeUri in the template. Make sure the binary exists in that folder you define.
  • The handler name should match the binary name. You should make sure these match.
  • It could be possible permissions are not set right on the binary itself. You should consult the AWS Lambda docs on permission that need to be set.

If you need further help, I encourage you to join our slack channel (details in the repo's README). This is a better place for support from the community and allows us to keep issue focused on Features and Bugs.

Closing as not reproducible.

@jfuss jfuss closed this as completed Mar 21, 2019
@charlieblevins
Copy link

charlieblevins commented Apr 9, 2019 •
edited
Loading

I had the exact same permissions issue but mine led to a panic nil pointer error. I solved it in my fork like so: charlieblevins/delve@3e1fa9c

Initial Issue:

Could not create config directory: mkdir /home/sbx_user1051: permission denied.panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x93bac9]

goroutine 1 [running]:
github.com/derekparker/delve/cmd/dlv/cmds.execute(0x0, 0xc0000bc780, 0x1, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/charlieblevins/go/src/github.com/derekparker/delve/cmd/dlv/cmds/commands.go:575 +0x3e9
github.com/derekparker/delve/cmd/dlv/cmds.New.func4(0xc000101b00, 0xc0000bc780, 0x1, 0x5)
	/Users/charlieblevins/go/src/github.com/derekparker/delve/cmd/dlv/cmds/commands.go:182 +0x65
github.com/derekparker/delve/vendor/github.com/spf13/cobra.(*Command).execute(0xc000101b00, 0xc0000cc480, 0x5, 0x8, 0xc000101b00, 0xc0000cc480)
	/Users/charlieblevins/go/src/github.com/derekparker/delve/vendor/github.com/spf13/cobra/command.go:647 +0x241
github.com/derekparker/delve/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000101200, 0xc000101200, 0xa71898, 0x5)
	/Users/charlieblevins/go/src/github.com/derekparker/delve/vendor/github.com/spf13/cobra/command.go:733 +0x2be
github.com/derekparker/delve/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/Users/charlieblevins/go/src/github.com/derekparker/delve/vendor/github.com/spf13/cobra/command.go:692
main.main()
	/Users/charlieblevins/go/src/github.com/derekparker/delve/cmd/dlv/main.go:24 +0xc7

The nil pointer is at line 575 of cmd/dlv/cmds/commands.go

I'm sure my forked solution is not the best as I do not fully understand the DebugInfoDirectories feature. The root cause is lacking permissions for the current linux user, which causes line 180 in pkg/config/config.go to fail. Since this really comes down to a permissions issue, I suspect the aws sam library should be notified, however at the very least it would be nice for delve to gracefully degrade if elevated permissions are not an absolute requirement.

@goldeneggg
Copy link
Contributor

I had the exact same permissions issue too.

Could not create config directory: mkdir /home/sbx_user1051: permission denied.

As far as I checked, delve try to create /home/sbx_user1051/.config/dlv directory here. But it will always fail because/home/sbx_user1051 directory has not existed yet at that time.

Actually, /home/sbx_user1051 directory is not created in Dockerfile of lambci/lambda-base
 and Dockerfile of lambci/lambda:build-go1.x.

@charlieblevins
Thank you for your detail information.

@m-david-lilly
Copy link

I'm having this problem and have not yet found a solution.

My steps:

mkdir golambda
cd golambda/
sam init --runtime go1.x
cd sam-app
code . # to launch VSCode
go get -u github.com/go-delve/delve/cmd/dlv
GOARCH=amd64 GOOS=linux go build -o ./dlv github.com/go-delve/delve/cmd/dlv
go get -u github.com/aws/aws-lambda-go/...
GOARCH=amd64 GOOS=linux go build -gcflags='-N -l' -o hello-world/hello-world ./hello-world
sam local start-api -d 5986 --debugger-path . --debug-args="-delveAPI=2" --debug

In a new terminal:
curl --silent --verbose http://127.0.0.1:3000/hello
Curl output:

*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /hello HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>

Curl hangs right here...

The output in the sam terminal:

2019-05-10 18:46:26 Using SAM Template at /Users/david.lilly/Work/go/src/golambda/sam-app/template.yaml
2019-05-10 18:46:26 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-10 18:46:26 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-10 18:46:26 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-10 18:46:26 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-10 18:46:26 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-10 18:46:26 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-10 18:46:26 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-10 18:46:26 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-10 18:46:26 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-05-10 18:46:26 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-10 18:46:26 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-05-10 18:46:26 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-10 18:46:26 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-10 18:46:26 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-10 18:46:26 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-10 18:46:26 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-10 18:46:26 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-10 18:46:26 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-10 18:46:26 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-10 18:46:26 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-05-10 18:46:26 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-10 18:46:26 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-05-10 18:46:26 local start-api command is called
2019-05-10 18:46:26 No Parameters detected in the template
2019-05-10 18:46:26 2 resources found in the template
2019-05-10 18:46:26 Found Serverless function with name='HelloWorldFunction' and CodeUri='hello-world/'
2019-05-10 18:46:26 Trying paths: ['/Users/david.lilly/.docker/config.json', '/Users/david.lilly/.dockercfg']
2019-05-10 18:46:26 Found file at path: /Users/david.lilly/.docker/config.json
2019-05-10 18:46:26 Couldn't find auth-related section ; attempting to interpret as auth-only file
2019-05-10 18:46:26 Config entry for key stackOrchestrator is not auth config
2019-05-10 18:46:26 http://localhost:None "GET /v1.35/_ping HTTP/1.1" 200 2
2019-05-10 18:46:26 No Parameters detected in the template
2019-05-10 18:46:26 2 resources found in the template
2019-05-10 18:46:26 Detected Inline Swagger definition
2019-05-10 18:46:26 Lambda function integration not found in Swagger document at path='/hello' method='get'
2019-05-10 18:46:26 Found '0' APIs in resource 'ServerlessRestApi'
2019-05-10 18:46:26 Found '1' API Events in Serverless function with name 'HelloWorldFunction'
2019-05-10 18:46:26 Removed duplicates from '0' Explicit APIs and '1' Implicit APIs to produce '1' APIs
2019-05-10 18:46:26 1 APIs found in the template
2019-05-10 18:46:26 Looking for credentials via: env
2019-05-10 18:46:26 Looking for credentials via: assume-role
2019-05-10 18:46:26 Looking for credentials via: shared-credentials-file
2019-05-10 18:46:26 Looking for credentials via: custom-process
2019-05-10 18:46:26 Looking for credentials via: config-file
2019-05-10 18:46:26 Looking for credentials via: ec2-credentials-file
2019-05-10 18:46:26 Looking for credentials via: boto-config
2019-05-10 18:46:26 Looking for credentials via: container-role
2019-05-10 18:46:26 Looking for credentials via: iam-role
2019-05-10 18:46:26 Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-05-10 18:46:26 Starting new HTTP connection (1): 169.254.169.254:80
2019-05-10 18:46:27 Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/iam/security-credentials/: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/botocore/utils.py", line 303, in _get_request
    response = self._session.send(request.prepare())
  File "/usr/local/lib/python2.7/site-packages/botocore/httpsession.py", line 282, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
2019-05-10 18:46:27 Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2019-05-10 18:46:27 Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/endpoints.json
2019-05-10 18:46:27 Event choose-service-name: calling handler <function handle_service_name_alias at 0x101ec0c80>
2019-05-10 18:46:27 Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/lambda/2015-03-31/service-2.json
2019-05-10 18:46:27 Event creating-client-class.lambda: calling handler <function add_generate_presigned_url at 0x101e6d230>
2019-05-10 18:46:27 The s3 config key is not a dictionary type, ignoring its value of: None
2019-05-10 18:46:27 Setting lambda timeout as (60, 60)
2019-05-10 18:46:27 Loading JSON file: /usr/local/lib/python2.7/site-packages/botocore/data/_retry.json
2019-05-10 18:46:27 Registering retry handlers for service: lambda
2019-05-10 18:46:27 Trying paths: ['/Users/david.lilly/.docker/config.json', '/Users/david.lilly/.dockercfg']
2019-05-10 18:46:27 Found file at path: /Users/david.lilly/.docker/config.json
2019-05-10 18:46:27 Couldn't find auth-related section ; attempting to interpret as auth-only file
2019-05-10 18:46:27 Config entry for key stackOrchestrator is not auth config
2019-05-10 18:46:27 Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
2019-05-10 18:46:27 You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2019-05-10 18:46:27 Localhost server is starting up. Multi-threading = False
2019-05-10 18:46:27  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
2019-05-10 18:47:53 Constructed String representation of Event to invoke Lambda. Event: {"body": null, "httpMethod": "GET", "resource": "/hello", "queryStringParameters": null, "requestContext": {"httpMethod": "GET", "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", "path": "/hello", "extendedRequestId": null, "resourceId": "123456", "apiId": "1234567890", "stage": "prod", "resourcePath": "/hello", "identity": {"accountId": null, "apiKey": null, "userArn": null, "cognitoAuthenticationProvider": null, "cognitoIdentityPoolId": null, "userAgent": "Custom User Agent String", "caller": null, "cognitoAuthenticationType": null, "sourceIp": "127.0.0.1", "user": null}, "accountId": "123456789012"}, "headers": {"X-Forwarded-Port": "3000", "Host": "127.0.0.1:3000", "X-Forwarded-Proto": "http", "Accept": "*/*", "User-Agent": "curl/7.54.0"}, "stageVariables": null, "path": "/hello", "pathParameters": null, "isBase64Encoded": false}
2019-05-10 18:47:53 Found one Lambda function with name 'HelloWorldFunction'
2019-05-10 18:47:53 Invoking hello-world (go1.x)
2019-05-10 18:47:53 Environment variables overrides data is standard format
2019-05-10 18:47:53 Loading AWS credentials from session with profile 'default'
2019-05-10 18:47:53 Looking for credentials via: env
2019-05-10 18:47:53 Looking for credentials via: assume-role
2019-05-10 18:47:53 Looking for credentials via: shared-credentials-file
2019-05-10 18:47:53 Looking for credentials via: custom-process
2019-05-10 18:47:53 Looking for credentials via: config-file
2019-05-10 18:47:53 Looking for credentials via: ec2-credentials-file
2019-05-10 18:47:53 Looking for credentials via: boto-config
2019-05-10 18:47:53 Looking for credentials via: container-role
2019-05-10 18:47:53 Looking for credentials via: iam-role
2019-05-10 18:47:53 Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2019-05-10 18:47:53 Starting new HTTP connection (2): 169.254.169.254:80
2019-05-10 18:47:54 Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/meta-data/iam/security-credentials/: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/botocore/utils.py", line 303, in _get_request
    response = self._session.send(request.prepare())
  File "/usr/local/lib/python2.7/site-packages/botocore/httpsession.py", line 282, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)
ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
2019-05-10 18:47:54 Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
2019-05-10 18:47:54 Resolving code path. Cwd=/Users/david.lilly/Work/go/src/golambda/sam-app, CodeUri=hello-world/
2019-05-10 18:47:54 Resolved absolute path to code is /Users/david.lilly/Work/go/src/golambda/sam-app/hello-world
2019-05-10 18:47:54 Code /Users/david.lilly/Work/go/src/golambda/sam-app/hello-world is not a zip/jar file
2019-05-10 18:47:54 Skipping building an image since no layers were defined
2019-05-10 18:47:54 Trying paths: ['/Users/david.lilly/.docker/config.json', '/Users/david.lilly/.dockercfg']
2019-05-10 18:47:54 Found file at path: /Users/david.lilly/.docker/config.json
2019-05-10 18:47:54 Couldn't find auth-related section ; attempting to interpret as auth-only file
2019-05-10 18:47:54 Config entry for key stackOrchestrator is not auth config
2019-05-10 18:47:54 http://localhost:None "GET /v1.35/images/lambci/lambda:go1.x/json HTTP/1.1" 200 None
2019-05-10 18:47:54 Looking for auth config
2019-05-10 18:47:54 No auth config in memory - loading from filesystem
2019-05-10 18:47:54 Trying paths: ['/Users/david.lilly/.docker/config.json', '/Users/david.lilly/.dockercfg']
2019-05-10 18:47:54 Found file at path: /Users/david.lilly/.docker/config.json
2019-05-10 18:47:54 Couldn't find auth-related section ; attempting to interpret as auth-only file
2019-05-10 18:47:54 Config entry for key stackOrchestrator is not auth config
2019-05-10 18:47:54 Looking for auth entry for 'docker.io'
2019-05-10 18:47:54 No entry found
2019-05-10 18:47:54 No auth config found
2019-05-10 18:47:55 http://localhost:None "POST /v1.35/images/create?tag=go1.x&fromImage=lambci%2Flambda HTTP/1.1" 200 None

Fetching lambci/lambda:go1.x Docker container image......
2019-05-10 18:47:55 Mounting /Users/david.lilly/Work/go/src/golambda/sam-app/hello-world as /var/task:ro,delegated inside runtime container
2019-05-10 18:47:55 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 90
2019-05-10 18:47:55 http://localhost:None "GET /v1.35/containers/44b607efc13da97ff9ccd09799060baafdcdfb36281ded64dd2ece28c2ec97b8/json HTTP/1.1" 200 None
2019-05-10 18:47:55 http://localhost:None "GET /v1.35/containers/44b607efc13da97ff9ccd09799060baafdcdfb36281ded64dd2ece28c2ec97b8/json HTTP/1.1" 200 None
2019-05-10 18:47:56 http://localhost:None "POST /v1.35/containers/44b607efc13da97ff9ccd09799060baafdcdfb36281ded64dd2ece28c2ec97b8/start HTTP/1.1" 204 0
2019-05-10 18:47:56 Setting up SIGTERM interrupt handler
2019-05-10 18:47:56 http://localhost:None "GET /v1.35/containers/44b607efc13da97ff9ccd09799060baafdcdfb36281ded64dd2ece28c2ec97b8/json HTTP/1.1" 200 None
2019-05-10 18:47:56 http://localhost:None "POST /containers/44b607efc13da97ff9ccd09799060baafdcdfb36281ded64dd2ece28c2ec97b8/attach?stream=1&stdin=0&logs=1&stderr=1&stdout=1 HTTP/1.1" 101 0
Could not create config directory: mkdir /home/sbx_user1051: permission denied.API server listening at: [::]:5986
2019-05-11T01:47:56Z info layer=debugger launching process with args: [/var/task/hello-world]
2019-05-10 18:48:56 Ignoring docker socket timeout
2019-05-10 18:49:56 Ignoring docker socket timeout
2019-05-10 18:50:56 Ignoring docker socket timeout
2019-05-10 18:51:56 Ignoring docker socket timeout

@m-david-lilly
Copy link

I updated my version of aws-sam-cli from 0.15.0 to 0.16.0 and it started working ...

@codephobia
Copy link

@goldeneggg Hey, running into the same issue. Did you manage to find a solution for the error?

@denisdanielyan
Copy link

denisdanielyan commented Jun 5, 2019 •
edited
Loading

sam: 0.17.0
go: 1.12.5
macos: 10.14.5

When following the steps from @jfuss sam hangs at the following line:

Could not create config directory: mkdir /home/sbx_user1051: permission denied.API server listening at: [::]:5986 2 2019-06-05T15:12:32Z info layer=debugger launching process with args: [/var/task/hello-world]
Also when running gops, i do not see the hello-world process.

The error message looks to be the same as @jfuss output but it is actually different:
My Version:
Could not create config directory: mkdir /home/sbx_user1051: permission denied.API server listening at: [::]:5986
@jfuss version:
Could not create config directory: mkdir .config: read-only file system.API server listening at: [::]:5986

Executing into the running container, I can verify that there is no /home/sbx_user1051 directory. And as suspected, /home belongs to root and the directory is not created as @goldeneggg has already pointed out.

I suspect that the issue does not arise from the missing home directory. Because when dlv fails to load / create it just returns a default config (see https://github.com/go-delve/delve/blob/master/pkg/config/config.go ln 69).

@jackmcguire1 jackmcguire1 mentioned this issue Aug 30, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@charlieblevins @goldeneggg @petrogko @denisdanielyan @codephobia @jackmcguire1 @jfuss @m-david-lilly