New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cli: Socket timed out without establishing a connection when --asset-parallelism=true #19930
Comments
@apoorvmote do you know what changed between when it used to work and now? Did you recently update |
Of course I am regularly changing CDK versions as it comes out. But in my opinion problem is not with CDK. But AWS is denying my specific IP address whenever I do large update for 30-50 functions. All the other stacks has very little update and it always goes through. Also I have another cdk project that I run in docker development environment and I am able to deploy over 50 functions without any problem from same computer. I can run that project in docker because all functions are nodejs built with esbuild. This (failing) project has functions written in golang and I use docker for building golang functions. I am not able to run docker inside docker so long story long I run this in WSL and deploy normally and it fails. But if I build it on WSL and docker then deploy from docker development environment because golang function is already built then it gets deployed. |
@apoorvmote can you try this with a version of the CDK |
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
I did upgrade to 2.22.0 and suddenly problem is disappeared. If the problem appears again then I will open another issue. |
|
It happened to me at CDK version |
Facing with this issue regularly now on 2.39.1 When I enable vpn and deploy again this error disappear so looks like this is somehow related to connection establishing issue. |
facing the same issue. I'm able to deploy several stacks that have lower resources but my lambdas aren't going through. Proxy seems to do the trick. System information: |
I get the same problem with the CDK 2.52.0 seems related to IP or connection somehow. Will get socket timeouts while trying to deploy:
Switch to another internet connection ( phone hot spot or similar ) and the problem goes away. System information: |
This does appear to be related to the Asset Parallelism feature. Executing a deployment with --asset-parallelism=false resulted in a successful deployment. When running without --asset-parallelism=false the stack failed on the following error:
There are only four assets in the bucket and none of them are over 50KB. System information: |
Thanks @jscrobinson I can confirm your finding! If I try to deploy normally it fails, but try again with the --asset-parallelism=false flag and the deployment succeeds. Just so happy to have a work around at the moment that doesn't involve finding a new internet connection. 😃 |
I've encountered the same issue using The workaround using |
We are running into this same issue when using the |
We also have to use the In general, a real solution for the underlying issue would be appreciated. In case it helps, we only see the problematic behavior when deploying from GitHub Actions. If we run the same deploy locally, it completes dramatically faster and without issues. So far, all our research regarding environment differences have been fruitless. |
This comment was marked as outdated.
This comment was marked as outdated.
Turns out our issue was caused by setting CDK is compiled into a single 28 MB Soon after, we realized that deployment performance was dramatically improved by upgrading to Node@20. This is due to this change in Node@19.6. Previously, we ran Node@18 LTS, which was also the highest supported version of CDK at the time. This change in Node@19.6 introduces caching for the parsed source maps, which resolves this whole problem entirely (for us). I stand by my point that the way CDK handles IO is ridiculous. I also think bundling a NodeJS module into a single 28 MB file, with a 58 MB source map is ridiculous. As Node@18 is also the latest supported runtime by AWS Lambda, be cautious when using p.s.: The reason it worked for us locally was, that nobody set |
😭 😭 😭 I encountered the same problem while deploying the project aws-samples/amazon-codewhisperer-workshop. And I tried using Log
Env
SolutionI switched the region from 😃 😃 😃 |
Describe the bug
I have anywhere between 20-50 nodejs lambda functions in single stack and I update their dependencies and deploy with cdk.
But lately I am not able to deploy updates. I get following error when I deploy.
I keep trying again and again and sometimes it goes through and most of the time it doesn't work.
Only stack with lower number of lambda functions sometimes gets deployed. But stack with large number of lambda functions fails 100% of the time.
Expected Behavior
I expected it to deploy no matter number of lambda functions in the stack. It used to get deployed without any problem.
Current Behavior
I don't know how to upgrade bootstrap version. I ran
cdk bootstrap
multiple times and it says no changes.Reproduction Steps
It was working before but suddenly stopped working.
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.20.0 (build 738ef49)
Framework Version
No response
Node.js Version
v16.14.2
OS
Ubuntu 20.04 on WSL 2
Language
Typescript
Language Version
~3.9.7
Other information
No response
The text was updated successfully, but these errors were encountered: