Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update broke Lambda Runners #420

Closed
QuinnyPig opened this issue Sep 13, 2023 · 2 comments
Closed

Update broke Lambda Runners #420

QuinnyPig opened this issue Sep 13, 2023 · 2 comments

Comments

@QuinnyPig
Copy link
Contributor

Just upgraded for the first time in half a year; now the Lambda runner is dying instantly with zero visibility, almost like the entrypoint is wrong. Help?

My code:

import * as cdk from 'aws-cdk-lib';
import { Size, Tags } from 'aws-cdk-lib';
import { Architecture, CodeBuildRunnerImageBuilder, CodeBuildRunnerProvider, GitHubRunners, LambdaRunnerProvider, Os } from '@cloudsnorkel/cdk-github-runners';

const app = new cdk.App();
const stack = new cdk.Stack(
  app,
  'github-runners',
  {
    env: {
      account: process.env.CDK_DEFAULT_ACCOUNT,
      region: process.env.CDK_DEFAULT_REGION,
    },
  },
);

const myBuilder = CodeBuildRunnerProvider.imageBuilder(stack, 'image builder', {
  architecture: Architecture.ARM64,
  os: Os.LINUX_UBUNTU
})

// create a custom Lambda provider
const myProvider = new LambdaRunnerProvider(stack, 'lambda runner', {
  labels: ['lambda-runner'],
  ephemeralStorageSize: Size.gibibytes(2),
  imageBuilder: myBuilder,
},

);

// create the runner infrastructure
new GitHubRunners(
  stack,
  'runners',
  {
    providers: [myProvider],
  }
);
Tags.of(app).add('project', 'lambda-runner');

app.synth();
@kichik
Copy link
Member

kichik commented Sep 13, 2023

The code in your post uses CodeBuildRunnerProvider.imageBuilder() instead of LambdaRunnerProvider.imageBuilder() so not all the required components are there. Specifically our wrapper that calls the runner from Lambda is not included and that will trigger Error: Runtime exited without providing a reason Runtime.ExitError as seen in the logs.

You can technically add the component yourself with the following, but it's best to use the right base image builder.

myBuilder.addComponent(RunnerImageComponent.lambdaEntrypoint());

There is also an issue where we expect you to use Os.LINUX_AMAZON_2 (or leave it out as it's the default). We always use Amazon Linux 2 base docker image right now.

Finally it looks like we're using node14 as the base image for this. That should definitely be updated.

The last two can be addressed with:

const myBuilder = LambdaRunnerProvider.imageBuilder(stack, 'image builder', {
  architecture: Architecture.ARM64,
  os: Os.LINUX_UBUNTU,
  baseDockerImage: 'a docker image that has ubuntu and lambda runtime (we use public.ecr.aws/lambda/nodejs:14-arm64 by default)',
});

mergify bot pushed a commit that referenced this issue Sep 13, 2023
Node 14.x is very EOL. Node 16.x should also be removed, but I tried upgrading to Node 18.x and got:

```
ERROR Cannot get required symbol EVP_rc2_cbc from libssl
ERROR ./config.sh: line 81: 204 Aborted (core dumped) ./bin/Runner.Listener configure "$@"
ERROR Invoke Error {"errorType":"Error","errorMessage":"Runner failed with exit code 134","stack":["Error: Runner failed with exit code 134"," at ChildProcess.<anonymous> (/var/task/runner.js:24:16)"," at ChildProcess.emit (node:events:514:28)"," at ChildProcess._handle.onexit (node:internal/child_process:291:12)"]}
```

See #420
@QuinnyPig
Copy link
Contributor Author

This got it! Possibly relevant to you: node16 is now apparently EOL. Ugh...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants