add --latest-only caching option #232

nickboldt · 2019-09-27T17:29:49Z

Change-Id:...

add --latest-only caching option

Change-Id: Ie0690a1ab8f9e89989b658a05b890075635c92e6
Signed-off-by: nickboldt nboldt@redhat.com

…/che does correct file assoc Change-Id: I4e12db0a173fc7aeb59947139ba39a01c6b227d0 Signed-off-by: nickboldt <nboldt@redhat.com>

…Dockerfile.caching Change-Id: Ib097d689276c68b45cac707c23efd2332ae7d6c4 Signed-off-by: nickboldt <nboldt@redhat.com>

Change-Id: I3c7d4f11774261b7110f44e58f6460440855b486 Signed-off-by: nickboldt <nboldt@redhat.com>

Change-Id: Ie0690a1ab8f9e89989b658a05b890075635c92e6 Signed-off-by: nickboldt <nboldt@redhat.com>

…nneeded USER commands Change-Id: Iff3aa7c7cc9b912b8f435e0932e8c0b21f5519bd Signed-off-by: nickboldt <nboldt@redhat.com>

nickboldt · 2019-09-27T17:47:55Z

BIggest change in this set is moving from Dockerfile.rhel (uses alpine + quay.io/openshiftio/rhel-base-httpd:latest) to rhel.Dockerfile (uses images from RHCC: ubi8-minimal and registry.access.redhat.com/rhscl/httpd-24-rhel7:2.4-104)

…iles Change-Id: I2b97cd814980d336b0b788e5526679d798ae2481 Signed-off-by: nickboldt <nboldt@redhat.com>

Dockerfile

README.md

rhel.Dockerfile

scripts/cache_artifacts.sh

rhel.Dockerfile

benoitf · 2019-09-30T14:18:38Z

why rhel.Dockerfile instead of Dockerfile.rhel ? it's hard to find it also it's not consistent with eclipse-che dockerfiles https://github.com/eclipse/che/tree/master/dockerfiles/che

rhel.Dockerfile

nickboldt · 2019-09-30T14:24:03Z

why rhel.Dockerfile instead of Dockerfile.rhel

the Dockerfile file extension is registered automatically in VSCode. The Dockerfile.* is not. This is why these files get autocompletion and linting:

Dockerfile
.Dockerfile
rhel.Dockerfile

And this does not:

Dockerfile.rhel

So to avoid bugs in the rhel version (which people are NOT using in upstream anyway, and aren't being run in CI) I'd rather at least make it easier to edit them :)

rhel.Dockerfile

…ded for Brew and the simpler '/usr/bin/pip3.6 install yq jsonschema' approach for online users; remove unneeded port, fix entry point Change-Id: I0c356dd93ebc714fc7534e86d8a398b97463c04d Signed-off-by: nickboldt <nboldt@redhat.com>

… .nix default paths) Change-Id: I5e9e15e955403b0fc1ffb97d28b1e1274673b883 Signed-off-by: nickboldt <nboldt@redhat.com>

…starts for @amisvek's advice Change-Id: I0d8c89776c3dcf30080dd43823b40c49cd01be60 Signed-off-by: nickboldt <nboldt@redhat.com>

Change-Id: I6ae8525d958eddbefa254fa371e93ed580dc869b Signed-off-by: nickboldt <nboldt@redhat.com>

…what doesn't work in downstream Change-Id: Ia1825e554135a7ca3463dbc27252b9ef9908cba3 Signed-off-by: nickboldt <nboldt@redhat.com>

nickboldt · 2019-09-30T20:52:05Z

two remaining questions are:

• about the .repo file (and reorging the gh repo itself to nest stuff under build/dockerfiles/ - which seems like a low-value change) and
• the question about a simple RUN rm -f ... vs. adding a --latest-only flag to the index.sh script so it can ignore non-latest content when enerating index.json

…g the meta.yaml too (OMG thanks @amisvek for the quick fix) Change-Id: If5ef526355a9c358af7b4a0c4ad917b1bc9528a8 Signed-off-by: nickboldt <nboldt@redhat.com>

benoitf · 2019-10-01T07:00:44Z

about Docker.rhel and rhel.Docker pattern I think we should not be driven by a VS Code extension.

also you've a way to make it working
microsoft/vscode-docker#192

amisevsk

Approving with some suggestions.

amisevsk · 2019-10-01T15:58:17Z

rhel.Dockerfile

+    fi && \
+    ln -s /usr/bin/python3.6 /usr/bin/python && \
+    # test install worked
+    for d in python yq jq jsonschema; do echo -n "$d: "; $d --version; done


In the eventual reorg, this could be pulled into a script that we copy in during the build to keep the dockerfile simpler.

amisevsk · 2019-10-01T16:02:34Z

rhel.Dockerfile

+
+# BEGIN these steps might not be required
+RUN sed -i /etc/httpd/conf/httpd.conf \
+    -e "s,Listen 80,Listen 8080," \


s,Listen 80,Listen 8080, is not required as the default is Listen 0.0.0.0:8080 (so there aren't matches), but the other lines do make changes (still don't know if they're necessary).

amisevsk · 2019-10-01T16:04:03Z

rhel.Dockerfile

+    -e "s,logs/error_log,/dev/stderr," \
+    -e "s,logs/access_log,/dev/stdout," \
+    -e "s,AllowOverride None,AllowOverride All," && \
+    chmod a+rwX /etc/httpd/conf /run/httpd


I believe these dirs are already access mode 777.

So you want me to remove this? or add -c to see what actually happens, if anything?

rhel.Dockerfile

scripts/uid_entrypoint.sh

Change-Id: Ic897685dd0873fc5d48e775fe15fd9ca4a55ddad Signed-off-by: nickboldt <nboldt@redhat.com>

amisevsk · 2019-10-01T16:08:16Z

Dockerfile

-RUN chmod -R g+rwX /build
+
+# To only cache files from /latest/ folders, use ./cache_artifacts.sh v3 --latest-only 
+# and uncomment line above to remove files so they're not included in index.json -- RUN rm -fr $(find /build/v3 -name 'meta.yaml' | grep -v "/latest/" | grep -o ".*/")


For the non-rhel dockerfile, this could be implemented as an ARG (ONLY_LATEST=true) to avoid having to edit the dockerfile for specific builds.

Change-Id: I64530e5a5cca77bbf2aabefc140bb1b6d8efe859 Signed-off-by: nickboldt <nboldt@redhat.com>

benoitf · 2019-10-01T20:02:01Z

rhel.Dockerfile

@@ -0,0 +1,129 @@
+#


Still a mistake to me to have rhel.Dockerfile instead of Dockerfile.rhel. It spread related files to different locations

benoitf · 2019-10-01T20:02:53Z

content_sets_epel7.repo

@@ -0,0 +1,5 @@
+[epel-7]


this file should be suffixes/prefixed by rhel to know it's only for rhel

benoitf · 2019-10-01T20:03:36Z

scripts/entrypoint.sh

@@ -0,0 +1,6 @@
+#!/bin/bash -x


It i missing copyright headers

benoitf · 2019-10-01T20:03:54Z

scripts/uid_entrypoint.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+if ! whoami &> /dev/null; then


Missing copyright headers

benoitf · 2019-10-01T20:04:17Z

scripts/uid_entrypoint.sh

+#!/bin/bash
+if ! whoami &> /dev/null; then
+  if [ -w /etc/passwd ]; then
+    echo "${USER_NAME:-jboss}:x:$(id -u):0:${USER_NAME:-jboss} user:${HOME}:/sbin/nologin" >> /etc/passwd


Why jboss ?

nickboldt added 4 commits September 27, 2019 13:28

switch from alpine+rhel to ubi+rhel-httpd; rename file so that vscode…

bd2a99c

…/che does correct file assoc Change-Id: I4e12db0a173fc7aeb59947139ba39a01c6b227d0 Signed-off-by: nickboldt <nboldt@redhat.com>

fix README to add new --target params and remove ref to non-existent …

accf1b2

…Dockerfile.caching Change-Id: Ib097d689276c68b45cac707c23efd2332ae7d6c4 Signed-off-by: nickboldt <nboldt@redhat.com>

combine RUN commands; add notes about --latest-only caching

f1b0158

Change-Id: I3c7d4f11774261b7110f44e58f6460440855b486 Signed-off-by: nickboldt <nboldt@redhat.com>

add --latest-only caching option

2fd9cc9

Change-Id: Ie0690a1ab8f9e89989b658a05b890075635c92e6 Signed-off-by: nickboldt <nboldt@redhat.com>

nickboldt requested review from amisevsk, l0rd, rhopp, skabashnyuk and vparfonov as code owners September 27, 2019 17:29

move LABEL and ENV stuff to the last image (needed by Brew); remove u…

eddf738

…nneeded USER commands Change-Id: Iff3aa7c7cc9b912b8f435e0932e8c0b21f5519bd Signed-off-by: nickboldt <nboldt@redhat.com>

fix positioning of the optional step to remove non-latest meta.yaml f…

277255d

…iles Change-Id: I2b97cd814980d336b0b788e5526679d798ae2481 Signed-off-by: nickboldt <nboldt@redhat.com>

nickboldt mentioned this pull request Sep 27, 2019

Enable filtering of old plugin versions from registry eclipse-che/che#14112

Closed

amisevsk reviewed Sep 30, 2019

View reviewed changes

benoitf reviewed Sep 30, 2019

View reviewed changes

rhel.Dockerfile Outdated Show resolved Hide resolved

benoitf reviewed Sep 30, 2019

View reviewed changes

rhel.Dockerfile Outdated Show resolved Hide resolved

nickboldt added 5 commits September 30, 2019 13:28

add entrypoint scripts to fix user and run httpd (works for different…

684fd80

… .nix default paths) Change-Id: I5e9e15e955403b0fc1ffb97d28b1e1274673b883 Signed-off-by: nickboldt <nboldt@redhat.com>

shaking a fist at circlei's shellcheck madness and thanking my lucky …

4047bb6

…starts for @amisvek's advice Change-Id: I0d8c89776c3dcf30080dd43823b40c49cd01be60 Signed-off-by: nickboldt <nboldt@redhat.com>

instead of including ENVs and LABELs, inject them downstream

0ff89b8

Change-Id: I6ae8525d958eddbefa254fa371e93ed580dc869b Signed-off-by: nickboldt <nboldt@redhat.com>

disable the artifact caching by default in upstream; add notes about …

4518097

…what doesn't work in downstream Change-Id: Ia1825e554135a7ca3463dbc27252b9ef9908cba3 Signed-off-by: nickboldt <nboldt@redhat.com>

delete more than just the meta.yaml files -- delete folders containin…

87d3db3

…g the meta.yaml too (OMG thanks @amisvek for the quick fix) Change-Id: If5ef526355a9c358af7b4a0c4ad917b1bc9528a8 Signed-off-by: nickboldt <nboldt@redhat.com>

amisevsk approved these changes Oct 1, 2019

View reviewed changes

add newline at end of script because shrug emoji

4633fd0

Change-Id: Ic897685dd0873fc5d48e775fe15fd9ca4a55ddad Signed-off-by: nickboldt <nboldt@redhat.com>

amisevsk reviewed Oct 1, 2019

View reviewed changes

accidentally commented out

0a65364

Change-Id: I64530e5a5cca77bbf2aabefc140bb1b6d8efe859 Signed-off-by: nickboldt <nboldt@redhat.com>

nickboldt merged commit a40609c into eclipse-che:master Oct 1, 2019

nickboldt mentioned this pull request Oct 1, 2019

[plugin registry] Improvements to build process eclipse-che/che#14736

Closed

benoitf reviewed Oct 1, 2019

View reviewed changes

content_sets_epel7.repo

@@ -0,0 +1,5 @@

[epel-7]

Copy link

Contributor

benoitf Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file should be suffixes/prefixed by rhel to know it's only for rhel

benoitf reviewed Oct 1, 2019

View reviewed changes

scripts/entrypoint.sh

@@ -0,0 +1,6 @@

#!/bin/bash -x

Copy link

Contributor

benoitf Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It i missing copyright headers

benoitf reviewed Oct 1, 2019

View reviewed changes

scripts/uid_entrypoint.sh

@@ -0,0 +1,7 @@

#!/bin/bash

if ! whoami &> /dev/null; then

Copy link

Contributor

benoitf Oct 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing copyright headers

benoitf reviewed Oct 1, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add --latest-only caching option #232

add --latest-only caching option #232

nickboldt commented Sep 27, 2019

nickboldt commented Sep 27, 2019

benoitf commented Sep 30, 2019

nickboldt commented Sep 30, 2019

nickboldt commented Sep 30, 2019

benoitf commented Oct 1, 2019

amisevsk left a comment

amisevsk Oct 1, 2019

amisevsk Oct 1, 2019

amisevsk Oct 1, 2019

nickboldt Oct 1, 2019

amisevsk Oct 1, 2019

benoitf Oct 1, 2019

benoitf Oct 1, 2019

benoitf Oct 1, 2019

benoitf Oct 1, 2019

benoitf Oct 1, 2019

add --latest-only caching option #232

add --latest-only caching option #232

Conversation

nickboldt commented Sep 27, 2019

nickboldt commented Sep 27, 2019

benoitf commented Sep 30, 2019

nickboldt commented Sep 30, 2019

nickboldt commented Sep 30, 2019

benoitf commented Oct 1, 2019

amisevsk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment