Move tools & configs from /home/user/ to /home/tooling/

[AObuchow]

This is a downstream port of devfile/developer-images#115, that fixes these downstream issues 1 and 2, with some adaptations made to account for some differences between the upstream and downstream UDI image, namely:

• The lack of a Red Hat packaged version of GNU stow: we are instead using cachito to build stow from it's GitHub mirror repo
• The use of bash's -e in the downstream UDI's entrypoint, which cause the entrypoint to fail if any command it executes has a non-zero return code
• The lack of sdkman and nvm on the downstream UDI
• The lack of vim on the downstream UDI

I also ported devfile/developer-images#117 while I was at it in afb8dfe.

I tried to organize the commits in this PR to mirror their upstream equivalents. Below is a list of downstream to upstream commit conversions:

• 42fae05 -> devfile/developer-images@48da97d
• 0bcddb6 -> devfile/developer-images@7ac7971
• 8eab81d -> devfile/developer-images@0b3a58d
• 9d328a6 -> devfile/developer-images@72f5307
• dbc7d1a -> devfile/developer-images@a9669fa
• aabe226 -> devfile/developer-images@18b26a7
• 07e4857 -> devfile/developer-images@0a20adb
• f828577 -> devfile/developer-images@d3b7f2e

The additional commits required for this change to work on the downstream UDI are as follows:

• Build stow from sources
• Install vim
• Ensure mountpoint test doesn't cause entrypoint to fail

A note on the repositories used in content_sets.yml: The Code Ready builder Beta repo is used for the s390x and ppc64le architectures. The non-beta version of the repo doesn't seem to exist.

Testing

This changes touches a lot of tools and configuration files used in the UDI. Thus, it's worth ensuring different workflows still function correctly (e.g. java, mvn, nodejs, go, python). In addition, ensuring podman (with the podman wrapper + kubedock) works correctly is important: see devfile/developer-images#107 for testing notes, though you'll have to modify this devfile to use the downstream UDI image built from this PR.

Comparison before and after changes

For reference, I'm comparing the UDI built from my changes with registry.redhat.io/devspaces/udi-rhel8:3.9-18

ls -la in /home/user/ directory

Before:

~ $ ls -la
total 16
drwxrwxr-x. 1 root root 184 Oct 31 22:49 .
drwxrwxr-x. 1 root root   8 Oct 31 22:41 ..
drwxrwxr-x. 1 root root  16 Oct 31 22:47 .ansible
-rw-rw-r--. 1 root root  89 Oct 31 22:45 .bashrc
drwxrwxr-x. 1 root root   6 Oct 31 22:45 .cache
drwxrwxr-x. 1 root root  20 Oct 31 22:41 .config
drwxrwxr-x. 1 root root  14 Oct 31 22:45 .java
drwxrwx---. 1 root root  12 Oct 31 22:47 .local
drwxrwxr-x. 1 root root  10 Oct 31 22:49 .npm
drwxrwxr-x. 1 root root  56 Oct 31 22:47 .venv
drwxrwxr-x. 1 root root  36 Oct 31 22:47 che
drwxrwxr-x. 1 root root   6 Oct 31 22:45 go
-rw-rw-r--. 1 root root 509 Oct 31 22:45 group.template
-rw-rw-r--. 1 root root 997 Oct 31 22:45 passwd.template
-rw-rw-r--. 1 root root 558 Oct 31 22:49 xdebug.log

After:

~ $ ls -la
total 28
drwxrwx---. 1 user root 242 Nov 13 23:19 .
drwxrwxr-x. 1 root root  22 Nov 13 23:14 ..
drwxrwxr-x. 1 root root  16 Nov 13 23:19 .ansible
-rw-rw-r--. 1 user root  18 Jun 20  2022 .bash_logout
-rw-rw-r--. 1 user root 141 Jun 20  2022 .bash_profile
-rw-rw-r--. 1 user root 376 Jun 20  2022 .bashrc
drwxrwxr-x. 1 root root   6 Nov 13 23:19 .cache
drwxrwxr-x. 1 root root  20 Nov 13 23:19 .config
drwxrwxr-x. 1 root root  14 Nov 13 23:19 .java
drwxrwxr-x. 1 root root  12 Nov 13 23:19 .local
drwxrwxr-x. 1 root root  56 Nov 13 23:19 .venv
-rw-rw----. 1 root root 532 Nov 13 23:19 .viminfo
drwxrwxr-x. 1 root root  36 Nov 13 23:19 che
drwxrwxr-x. 1 root root   6 Nov 13 23:19 go
lrwxrwxrwx. 1 root root  25 Nov 13 23:19 group.template -> ../tooling/group.template
lrwxrwxrwx. 1 root root  26 Nov 13 23:19 passwd.template -> ../tooling/passwd.template
lrwxrwxrwx. 1 root root  21 Nov 13 23:19 xdebug.log -> ../tooling/xdebug.log

/home/user/ directory tree

Directory tree can be viewed using https://github.com/a8m/tree/tree/master: installed by running go install github.com/a8m/tree/cmd/tree@latest then go clean -cache && go clean -modcache && rm -rf /home/user/.cache/go-build && rm -rf /home/user/.cache/pip

Unfortunately, the trees are too large to fit within a PR description.

.bashrc

Before:

~ $ cat .bashrc
export PS1='\W `git branch --show-current 2>/dev/null | sed -r -e "s@^(.+)@\(\1\) @"`$ '

After:

~ $ cat .bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi

# User specific environment
if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
then
    PATH="$HOME/.local/bin:$HOME/bin:$PATH"
fi
export PATH

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions

Also, here are the 2 new files in /etc/profile.d/:

~ $ cat /etc/profile.d/udi_environment.sh  
alias docker=podman  
~ $ cat /etc/profile.d/udi_prompt.sh
export PS1='\W `git branch --show-current 2>/dev/null | sed -r -e "s@^(.+)@\(\1\) @"`$ '

$PATH

Before:

~ $ echo $PATH | tr : \\n | sort
/bin
/home/user/.java/current/bin
/home/user/.local/bin
/home/user/.npm-global/bin/
/home/user/.venv/bin
/home/user/go/bin
/home/user/node_modules/.bin/
/opt/app-root/src/.npm-global/bin/
/sbin
/usr/bin
/usr/bin
/usr/local/bin
/usr/local/sbin
/usr/sbin
/usr/share/maven/bin

After:

~ $ echo $PATH | tr : \\n | sort
/bin
/home/tooling/.java/current/bin
/home/tooling/.local/bin
/home/tooling/.npm-global/bin/
/home/tooling/.venv/bin
/home/tooling/go/bin
/home/tooling/node_modules/.bin/
/home/user/.dotnet/tools
/home/user/.local/bin
/home/user/.local/bin
/home/user/bin
/opt/app-root/src/.npm-global/bin/
/sbin
/usr/bin
/usr/bin
/usr/local/bin
/usr/local/sbin
/usr/sbin
/usr/share/Modules/bin
/usr/share/maven/bin

[svor]

@AObuchow do you have a built image that contains current changes, I'd like to test it with existing DS samples

[AObuchow]

@svor here's an image built from these changes: registry-proxy.engineering.redhat.com/rh-osbs/devspaces-udi-rhel8:devspaces-3-rhel-8-containers-candidate-87113-20231114040246-x86_64. You'll need to authenticate in order to pull it, however. If you have any issues, let me know

[AObuchow]

As discussed in standup, I will also include an extra commit to backport devfile/developer-images#135 to the downstream UDI

[nickboldt]

probably OK, but some nitpicks noted could be revised.

[AObuchow]

I pushed additional commits based on the feedback so far. @svor @nickboldt thank you both for the reviews.

b8ecb9a is a port of devfile/developer-images#135, with Gradle and SBT being excluded since they are not being installed on the downstream UDI.

The newest changes are available for testing at: registry-proxy.engineering.redhat.com/rh-osbs/devspaces-udi-rhel8:devspaces-3-rhel-8-containers-candidate-68595-20231114193307-x86_64

[AObuchow]

It might actually block shipping the image later in general -- if you check those URLs, texinfo and texinfo-tex are only available on x86_64 and aarch64

Good catch @amisevsk. I'm not sure why I was under the impression texinfo and texinfo-tex were available in the Code Ready Beta repo's.

Might be safer to only do the stow stuff if the arch is supported

Not sure yet what the cleanest and most feasible way to do this is: do we create a second dockerfile for PPC & s390x? Do we do a runtime check on the architecture?

It's probably worth investigating if there's a simple way to disable the need for tex when building stow. Will look into that further.

[nickboldt]

cleanest and most feasible way to do this is: do we create a second dockerfile for PPC & s390x?

100% no, that won't work

Do we do a runtime check on the architecture?

Yes, we do this when support on various arches varies, like the versions of dotnet available on Z & P:

https://github.com/redhat-developer/devspaces-images/blob/devspaces-3-rhel-8/devspaces-udi/Dockerfile#L284-L290

So you would want to ONLY include stow on x64, unless you can figure out a way to build it on Z & P too.

You can ask for help on those arches from @Sachin-Itagi or @ghatwala, if you want to just implement this for x64 and ignore the other two arches for now.

Either way, we'll likely need to loop in @deerskindoll to add a new RNaKI item if the support is different by arch.

[svor]

@AObuchow it's not possible to build a java project with current changes, when executing mvn clean install. I didn't investigate what is the reason, but I see an error:

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.092 s
[INFO] Finished at: 2023-11-15T13:27:58Z
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.5: Could not transfer artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.5 from/to central (https://repo.maven.apache.org/maven2): No connector factories available -> [Help 1]

You can try it with lombok DS sample:

Also a problem with dotnet:

 *  Executing task: devfile: 1.Update dependencies 

sh: dotnet: command not found

You can try it with

And I didn't detect any problem with python, nodejs, go and php

[l0rd]

I have did some tests building, running containers and switching java version and everything worked just fine. LGTM 👍

[AObuchow]

@svor Thank you for the feedback, I'll be working on reproducing and trying to fix the issues with maven & dotnet that you described.

[AObuchow]

@l0rd Glad to hear things seemed to work correctly in your testing.

[AObuchow]

Do we do a runtime check on the architecture?

Yes, we do this when support on various arches varies, like the versions of dotnet available on Z & P:

devspaces-3-rhel-8/devspaces-udi/Dockerfile#L284-L290

So you would want to ONLY include stow on x64, unless you can figure out a way to build it on Z & P too.

Thank you for the guidance @nickboldt.

Something strange to note: my brew builds have actually been succeeding (including s390x & PPC). Looking at the build logs, I can see that stow seems to be getting built, and its version is being outputted correctly after building: stow (GNU Stow) version 2.3.2.

In the logs I see that there is a texinfo-tex package for both architectures (texinfo-tex-6.5-7.el8.ppc64le & texinfo-tex-6.5-7.el8.s390x) and a texinfo package (texinfo-6.5-7.el8.ppc64le & texinfo-6.5-7.el8.s390x).

This doesn't mean we're in the clear yet as we're still using the Code Ready Beta repo's however.

[AObuchow]

@svor @nickboldt For dotnet, it seems that the install is failing:

2023-11-14 19:49:31,920 - atomic_reactor.tasks.binary_container_build - INFO - Error:
2023-11-14 19:49:31,920 - atomic_reactor.tasks.binary_container_build - INFO -  Problem: cannot install the best candidate for the job
2023-11-14 19:49:31,920 - atomic_reactor.tasks.binary_container_build - INFO -   - nothing provides netstandard-targeting-pack-2.1(x86-64) >= 7.0.113-2.el8_9 needed by dotnet-sdk-7.0-7.0.113-2.el8_9.x86_64 from rhel-8-for-x86_64-appstream-rpms

I'm not sure yet why this is happening. Adding an extra repository shouldn't cause this issue? Unless a subset of alternative packages were mistakenly installed from the additional Code Ready builder repo, and their matching dependencies (required for dotnet) aren't present. Maybe I need to disable the code ready repo after stow has been built?

Edit: it seems like the UDI has dotnet-7.0.112-1.el8_8.x86_64 installed, and my brew build is trying to install dotnet-7.0.113-1.el8_8.x86_64 instead.

[AObuchow]

@svor mvn and dotnet should now be fixed.

For maven, the wrong maven-resolver package was accidentally getting pulled from the code ready builder repo so I disabled it after building stow (though, maybe this should happen earlier in the dockerfile.. it looks kind of ugly, but I wasn't sure about introducing an extra layer).

For dotnet, I think there was some weird upgrade that was in-flight for dotnet's dependencies when I built the image. It seems to work now.

@amisevsk I addressed your feedback. I realized certain changes belonged to the wrong commits, which is why you see me deleting and re-adding the same lines (with slight modifications based on your feedback).

[AObuchow]

Something that came to mind: maven is using /root/.m2 as the local repository location to download artifacts. Ideally it should be /home/user/.m2, but I haven't found the right environment variable to modify this behaviour, and the default settings.xml mentions that the default local maven repository should be $HOME/.m2 (but that is not the case ?)

Also, we might want to change GOBIN and GOPATH to point to /home/user/go/ instead of /home/tooling/go so that go modules that are downloaded persist on the PVC if persistUserHome is enabled. I seem to have missed this detail when making the upstream PR for this change. @l0rd any thoughts on this?

[AObuchow]

The latest changes can be tested with registry-proxy.engineering.redhat.com/rh-osbs/devspaces-udi-rhel8:devspaces-3-rhel-8-containers-candidate-68148-20231116060632-x86_64

[AObuchow]

@amisevsk I added 2 extra commits to address the PATH issue and code-ready repo disable. Though I am still waiting on my brew build to finish.

[AObuchow]

My fix for the PATH problem was incorrect... sorry about that. Fixing it.

[AObuchow]

The latest changes can be tested with: docker run -ti --rm registry-proxy.engineering.redhat.com/rh-osbs/devspaces-udi-rhel8:devspaces-3-rhel-8-containers-candidate-43055-20231120202203-x86_64 bash

[AObuchow]

I'm currently testing out the image in a Che workspace with persistUserHome enabled and something is unfortunately failing in the entrypoint, looking into it.

Edit: Looks it stow is failing to due the ~/.java/current/ dir:

Planning stow of package .... done
WARNING! stowing . would cause conflicts:
  * existing target is not owned by stow: .java/current/bin
  * existing target is not owned by stow: .java/current/conf
  * existing target is not owned by stow: .java/current/include
  * existing target is not owned by stow: .java/current/legal
  * existing target is not owned by stow: .java/current/lib
  * existing target is not owned by stow: .java/current/release
  * existing target is not owned by stow: .java/current/tapset
All operations aborted.

Edit 2: Should be fixed in 49b87a9 (see commit description). I pushed my image (registry-proxy.engineering.redhat.com/rh-osbs/devspaces-udi-rhel8:devspaces-3-rhel-8-containers-candidate-90312-20231121002243-x86_64) to quay at quay.io/aobuchow/universal-developer-image:downstream.

I tested things in Che with the following devfile on the dogfooding cluster, and modified the devworkspace object to use the following dwoc with persistUserHome enabled:

apiVersion: controller.devfile.io/v1alpha1
config:
  workspace:
    persistUserHome:
      enabled: true
kind: DevWorkspaceOperatorConfig
metadata:
  name: custom-dwoc
  namespace: aobuchow-che-1cac83

I did a quick test that involved:

• maven build devfile task to make sure the .m2 repo was being used and persisted (after workspace restart ~ though I had to re-modify the devworkpace object to use my dwoc)
• installed go imports and made sure it persisted workspace restart
• ran a few tools: make, pip, python, node & dotnet
• Switched to Java 17 by setting $USE_JAVA17=true and re-running the entrypoint.

[AObuchow]

@nickboldt If this gets a +1 from @amisevsk & @SDawley, I will squash the remaining fixup commits and this should be good to finally merge 🥳

[devstudio-release]

Build 3.11 :: udi_3.x/389: Console, Changes, Git Data

[devstudio-release]

Build 3.11 :: get-sources-rhpkg-container-build_3.x/5203: Console, Changes, Git Data

[devstudio-release]

Build 3.11 :: get-sources-rhpkg-container-build_3.x/5203:

udi : 3.x :: Failed in 57108217 : BREW:BUILD/STATUS:UNKNOWN
FAILURE:; copied to quay