CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux #101

anien · 2022-11-21T13:04:30Z

Originally, enclaves were built in the docker container only on macOS and Windows. Now, all GraalVM enclaves are built in the conclave-build container regardless the OS. Gramine enclaves are still built outside this container, therefore python, Gramine, etc still do need to be installed on the machines.

Docker is now installed in the integration-tests docker container. Conclave-build container is now run with the same user as the integration-tests container - this is to prevent permission issues when deleting files.

…iner

…va Gramine enclaves

shamsasari · 2022-11-23T17:53:59Z

Caused by: net.rubygrapefruit.platform.NativeException: Could not start 'docker'
14:59:01     at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:27)
14:59:01     at net.rubygrapefruit.platform.internal.WrapperProcessLauncher.start(WrapperProcessLauncher.java:36)
14:59:01     at org.gradle.process.internal.ExecHandleRunner.startProcess(ExecHandleRunner.java:98)
14:59:01     at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:71)
14:59:01     ... 7 more
14:59:01   Caused by: java.io.IOException: Cannot run program "docker" (in directory "/data/buildAgent/work/a4c864046b3e21f/integration-tests/general/threadsafe-enclave"): error=2, No such file or directory
14:59:01     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
14:59:01     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
14:59:01     at net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:25)
14:59:01     ... 10 more
14:59:01   Caused by: java.io.IOException: error=2, No such file or directory
14:59:01     at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
14:59:01     at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
14:59:01     at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
14:59:01     at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)

It can't find docker, which doesn't make sense, until you realise this is inside the integration-tests container! So perhaps we have install docker inside that container??

anien · 2022-11-24T09:40:49Z

Docker is available inside integrations-tests container. We already expose docker socket to the container by having
-v /var/run/docker.sock:/var/run/docker.sock parameter.
According this: https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/, that option allows you to create a 'sibling' containers, instead of a child containers, which is better approach. The Could not start 'docker' error is just hiding the real problem. If you add --privileged to docker command (which the article above recommend as well), the error changes to Permission denied. We do funky things with users in our docker containers, which I don't understand yet. I think we now only need to add some users we have in the docker container to the docker group inside the integration-tests container, so it can start the docker process.

...adle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/GenerateEnclaveSigningMaterial.kt

...-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/GradleEnclavePlugin.kt

plugin-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/LinuxExec.kt

shamsasari · 2022-11-24T13:40:45Z

...c/main/kotlin/com/r3/conclave/plugin/enclave/gradle/gramine/GenerateGramineDirectManifest.kt

-            .splitToSequence("\n")
-            .single { it.startsWith("Location: ") }
-            .substringAfter("Location: ")
+        //  non-python. https://r3-cev.atlassian.net/browse/CON-1181 and https://r3-cev.atlassian.net/browse/CON-1229.


Only the python bits are left from this TODO, so update accordingly. No need for the CON-1181 reference.

You still need python, pip and jep libraries installed on a local machine for Python enclaves. Updated accordingly.

...c/main/kotlin/com/r3/conclave/plugin/enclave/gradle/gramine/GenerateGramineDirectManifest.kt

.../src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/gramine/GenerateGramineSGXManifest.kt

…cker

shamsasari · 2022-12-02T12:04:40Z

Yay, the integration tests now pass! Now to add that flag and update the error message to point to the GItHub issues page (and merge conflict!)

shamsasari

Some more comments, hopefully the last.

plugin-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/LinuxExec.kt

...c/main/kotlin/com/r3/conclave/plugin/enclave/gradle/gramine/GenerateGramineDirectManifest.kt

...radle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/gramine/GenerateGramineBundle.kt

plugin-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/LinuxExec.kt

shamsasari · 2022-12-06T16:26:49Z

plugin-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/LinuxExec.kt

@@ -40,11 +44,14 @@ open class LinuxExec @Inject constructor(objects: ObjectFactory) : ConclaveTask(
                    conclaveBuildDir
                )
            } catch (e: Exception) {
+                logger.info("Docker build of conclave-build failed. Re-run your build with '--info' and " +
+                        "submit a new GitHub issue https://github.com/R3Conclave/conclave-core-sdk/issues/new", e)


I don't think we need to mention submitting a GH issue here. It could be they find out what the issue is by enabling --info and fix the issue themselves. Mentioning it in the exception message is sufficient.

Hang on, we've got this wrong! The mention of the --info flag needs to be in the exception below.

Okay, that now makes sense :) I'll fix that.

Hopefully this now make sense.

shamsasari · 2022-12-07T09:38:00Z

Can you update PR description.

shamsasari

Looking good!

@JohnJoser3 remind me that we need to sit down regarding documentation for this and all the Gramine stuff.

#101 accidently made building on MacOS and Windows conditional on the new `buildInDocker` flag. This is not correct as Docker is always required for these environments.

anien added 7 commits November 10, 2022 16:05

First try to use docker when building enclave

57d8273

Create gramine manifest in the docker container

4c6bcae

Merge master

2aad39c

Make simulation mode with gramine working in the conclave-build conta…

e05b1f2

…iner

Build python enclaves outside of the container

d40f024

Run docker command with retrieving output

f705793

Merge master

4c65d81

anien marked this pull request as draft November 21, 2022 13:04

anien added 5 commits November 21, 2022 15:13

Use commandWithOutput when retrieving output from a docker command

9b5d122

Improve comments

df37914

Do not require gcc and python to be installed on Linux if building Ja…

4a25b4d

…va Gramine enclaves

Formatting

9e058e4

More formatting

feeb461

anien marked this pull request as ready for review November 23, 2022 11:49

anien requested review from shamsasari and bon000 November 23, 2022 11:50

Add stack trace to Docker build failure.

6455232

shamsasari suggested changes Nov 24, 2022

View reviewed changes

anien added 5 commits November 28, 2022 10:45

Install doker in integration-tests docker container

97c7360

Merge master

8f6e903

Assign to val directly

7fe3afb

Change comment to TODO, add jira ticket link

ccd9304

Add TODO

7566881

anien force-pushed the anna/CON-1203-1 branch from 65ecc19 to 7566881 Compare November 28, 2022 23:49

anien added 3 commits November 30, 2022 10:23

Update comment and do not duplicate code

2a9f36a

Update comment about docker RAM, avoid duplicate code when calling do…

6aa61fa

…cker

Update gitignore

1395139

anien requested a review from shamsasari November 30, 2022 11:13

anien added 4 commits December 1, 2022 15:13

Try to hardcode user when running docker

755a1f3

Retrieve user and group properly

701c86f

Merge master

266f569

trim end

147ba4c

anien added 8 commits December 5, 2022 13:13

Add flag buildInDocker to disable building enclave in docker

4be9134

Revert few changes and put back missing bits

ce72aac

Return docker params if buildInDocker is true

5880cd2

Update condition

88a2f27

Merge master

5311323

Merge master

2c41529

Revert changes to Gramine enclaves

68994fa

Remove new line

872d015

shamsasari reviewed Dec 6, 2022

View reviewed changes

anien added 3 commits December 6, 2022 15:36

Add missing buildInDocker setup

17a4bf7

Address review comments

678636c

Formatting

8dcd82a

shamsasari reviewed Dec 6, 2022

View reviewed changes

plugin-enclave-gradle/src/main/kotlin/com/r3/conclave/plugin/enclave/gradle/LinuxExec.kt Outdated Show resolved Hide resolved

shamsasari reviewed Dec 6, 2022

View reviewed changes

anien added 2 commits December 6, 2022 16:40

Address review comments

0a1df70

Address another set of review comments

e246176

anien requested a review from shamsasari December 6, 2022 23:04

shamsasari changed the title ~~CON-1203: Gradle plugin to use conclave-build container on Linux~~ CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux Dec 7, 2022

Prepare files correctly when building in docker

5892424

shamsasari approved these changes Dec 7, 2022

View reviewed changes

github-actions bot merged commit 4dc73d2 into master Dec 7, 2022

github-actions bot deleted the anna/CON-1203-1 branch December 7, 2022 12:08

shamsasari mentioned this pull request Dec 16, 2022

CON-1203: Non-Linux envs must always use Docker for building the enclave #125

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux #101

CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux #101

anien commented Nov 21, 2022 •

edited by shamsasari

Loading

shamsasari commented Nov 23, 2022 •

edited

Loading

anien commented Nov 24, 2022

shamsasari Nov 24, 2022

anien Nov 28, 2022

shamsasari commented Dec 2, 2022

shamsasari left a comment

shamsasari Dec 6, 2022

shamsasari Dec 6, 2022

anien Dec 6, 2022

anien Dec 6, 2022

shamsasari commented Dec 7, 2022

shamsasari left a comment

CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux #101

CON-1203: Gradle plugin using conclave-build container for GraalVM on Linux #101

Conversation

anien commented Nov 21, 2022 • edited by shamsasari Loading

shamsasari commented Nov 23, 2022 • edited Loading

anien commented Nov 24, 2022

shamsasari Nov 24, 2022

Choose a reason for hiding this comment

anien Nov 28, 2022

Choose a reason for hiding this comment

shamsasari commented Dec 2, 2022

shamsasari left a comment

Choose a reason for hiding this comment

shamsasari Dec 6, 2022

Choose a reason for hiding this comment

shamsasari Dec 6, 2022

Choose a reason for hiding this comment

anien Dec 6, 2022

Choose a reason for hiding this comment

anien Dec 6, 2022

Choose a reason for hiding this comment

shamsasari commented Dec 7, 2022

shamsasari left a comment

Choose a reason for hiding this comment

anien commented Nov 21, 2022 •

edited by shamsasari

Loading

shamsasari commented Nov 23, 2022 •

edited

Loading