New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
projen workflows cannot run docker commands by default #2094
Comments
Tests need docker access actually, needed to follow instructions here projen/projen#2094 to enable this.
I propose |
Thank you so much @kaizencc you just saved me heaps of time here. Googled your post, applied the fixes, and everything now runs in GitActions, and my cdk-rds-sql library has now turned up on Construct Hub. Couldn't have done this without projen in this short amount of time. |
What worked for me, was to add the docker group. Unfortunately this is just a number and not a nice name. I guess it might change and break again without warning. const buildWorkflow = project.tryFindObjectFile('.github/workflows/build.yml');
buildWorkflow.patch(JsonPatch.add('/jobs/build/container/options', '--group-add 121')); |
Finally open an issue with the GitHub team |
Update: I found the best workaround for this was to simply not use the superchain image: https://github.com/mrgrain/streamlink-serverless/blob/main/projenrc/WorkflowNoDockerPatch.ts Additionally requires Node.js to be set-up, but that's okay. Why does this work? projen doesn't actually run any non-Node.js tasks in the main build. So there really is no inherent advantage of using the superchain image for this. All the subsequent packaging and release jobs already setup the required tools as needed. I'm strongly considering to make this the default. Particularly since running inside Docker also suffers from the issue that Docker bundling won't work (see aws/aws-cdk#8799) |
I just committed this file [here](#54), and messed something up. Using this workaround: projen/projen#2094 (comment)
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
… in workflows BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using `jsii/superchain` provides no tangible benefit over installing dependencies with GitHub Actions. However because AWS CDK Constructs often require to run docker commands, with image GitHub Action workflows end up attempting to run Docker in Docker. This is not trivial to achieve (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing further problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065
…flows (#2510) BREAKING CHANGE: default to node16 To use any other Node version, explicitly provide the desired version number BREAKING CHANGE: remove `jsii/superchain` image from AwsCdkConstructLibrary workflows Using the `jsii/superchain` image provides no tangible benefit over installing dependencies with GitHub Actions. However AWS CDK Constructs often require to run docker commands, and thus using the image forces GitHub Action to exectute commands as Docker in Docker. This is does not work in many situations and generic fixes are not reliable (see #2094 & aws/aws-cdk#8799). Additionally the existing build and package workflows had an inconsistent usage of the image, causing more problems. To restore the old behavior, set `options.workflowContainerImage` to the desired image. Fixes #2094 Closes #1065 --- By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Problem:
Sometimes docker commands are bundled into projen commands. For example, in https://github.com/cdklabs/awscdk-asset-awscli, we specify a pre-compile step that executes a
build.sh
script. This script runs adocker build
command, which builds a local image. This encounters the following problem:Under the hood, projen by default is using the
jsii/superchain:1-buster-slim-node14
container image and docker commands are not set up to be run inside this image. Therefore, the following solution is necessary:Which gives the
superchain
user ownership of the docker socket.Proposal:
This is a sharp edge that we had to figure out, and I can't imagine there aren't other scenarios where we want to run docker commands in a projen-managed repository. The proposal is to surface this command as a boolean property on the projen library construct, like so:
And, because of aws/jsii#3746, configuring passwordless sudo requires additional changes to
.projenrc.js
. We have been able to work around this via projen escape hatches:Until the issue in jsii is somehow addressed, this will need to be baked into this proposal as well.
The text was updated successfully, but these errors were encountered: