All bitnami stacks do not work

[artaleks9]

Reproduction Steps:

• Go to the dashboard.
• Try to create a template project from any bitnami stack in the stack library.

Expected behavior:

• The project should be created and opened in the IDE.

Observed behavior:

• The workspace agent does not start.

Che version: 5.1.0-SNAPSHOT
OS and version: Ubuntu 14.04
Docker version: 1.12

Additional information:

• Problem can be reliably reproduced: Yes
• See the attachment:
  

[ghost]

Must be related to this commit - 2eb57f0

[ghost]

Also, @artaleks9 please provide logs - it'll be easier for engineers to investigate:

[STDERR] /bin/sh: 46: [: Illegal number: /bin/sh: 45: /bin/java: not found
[STDOUT] Hit http://security.debian.org jessie/updates InRelease
[STDOUT] Ign http://httpredir.debian.org jessie InRelease
[STDOUT] Get:1 http://security.debian.org jessie/updates/main amd64 Packages [431 kB]
[STDOUT] Hit http://httpredir.debian.org jessie Release.gpg
[STDOUT] Hit http://httpredir.debian.org jessie Release
[STDOUT] Get:2 http://httpredir.debian.org jessie/main amd64 Packages [9049 kB]
[STDOUT] Fetched 9480 kB in 10s (929 kB/s)
[STDOUT] Reading package lists...
[STDOUT] Reading package lists...
[STDOUT] Building dependency tree...
[STDOUT] Reading state information...
[STDERR] E: Unable to locate package openjdk-8-jdk-headless
[STDOUT] Workspace Agent will be downloaded from Workspace Master
[STDERR] /home/bitnami/che/ws-agent/bin/catalina.sh: 1: exec: /home/bitnami/che/jdk1.8/bin/java: not found

In this case, package openjdk-8-jdk-headless isn't found.

I have tried installing this package on a virgin debian:jessie and failed to do so:

C:\Users\Codenvy>docker pull debian:jessie
jessie: Pulling from library/debian

75a822cd7888: Pull complete
Digest: sha256:f7062cf040f67f0c26ff46b3b44fe036c29468a7e69d8170f37c57f2eec1261b
Status: Downloaded newer image for debian:jessie

C:\Users\Codenvy>docker run -ti debian:jessie sh
# apt-get update
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Get:2 http://security.debian.org jessie/updates/main amd64 Packages [431 kB]
Ign http://deb.debian.org jessie InRelease
Get:3 http://deb.debian.org jessie-updates InRelease [145 kB]
Get:4 http://deb.debian.org jessie Release.gpg [2373 B]
Get:5 http://deb.debian.org jessie Release [148 kB]
Get:6 http://deb.debian.org jessie-updates/main amd64 Packages [17.6 kB]
Get:7 http://deb.debian.org jessie/main amd64 Packages [9049 kB]
Fetched 9856 kB in 14s (692 kB/s)
Reading package lists... Done
# apt-get install openjdk-8-jdk-headless
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package openjdk-8-jdk-headless

So, this package is unavailable in debian out of the box. What did the trick is:

echo  "deb http://ftp.de.debian.org/debian jessie-backports main" >> /etc/apt/sources.list
sudo apt-get update

Therefore, this section of the script should be changed:

https://github.com/eclipse/che/blob/master/agents/che-core-api-agent/src/main/resources/agents/scripts/org.eclipse.che.ws-agent.script.sh#L95-L110

[ghost]

The fix worked for a while, now getting:

[STDOUT] openjdk-8-jdk-headless : Depends: openjdk-8-jre-headless (= 8u121-b13-1~bpo8+1) but it is not going to be installed
[STDERR] E: Unable to correct problems, you have held broken packages.

[riuvshin]

this is how I reproduced issue whithout codenvy on pure debian:8 image

docker run -ti debian:8 bash
root@89090aca0bf7:/# echo "deb http://httpredir.debian.org/debian jessie-backports main" | ${SUDO} tee --append /etc/apt/sources.list
root@89090aca0bf7:/# apt-get update
root@89090aca0bf7:/# 
root@89090aca0bf7:/# apt-get install openjdk-8-jdk-headless

 openjdk-8-jdk-headless : Depends: openjdk-8-jre-headless (= 8u121-b13-1~bpo8+1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages

So this is pure debian and their infra issue, to eliminate this in future I would recommend to preinstall in all stacks everything that is mandatory for workspace start like jdk rsync etc.

[bmicklea]

Good investigation guys - a few questions:

• if we're waiting on a fix from Debian is there a Debian issue we can link to?
• is there anything we / Bitnami can do as a workaround?
• I think this is a P1 would anyone disagree?

[riuvshin]

I don;t think we can do something with this issue. I think there is no reason to keep it even open. This is on debian side and we have no info about how and when they will fix this.

to workaround this I've already proposed for Bitnami guys

So this is pure debian and their infra issue, to eliminate this in future I would recommend to preinstall in all stacks everything that is mandatory for workspace start like jdk rsync etc.

[james-w]

Hi,

I've looked in to why this is happening.

The short version is that apt is configured to not automatically take packages from the
backports repo unless you tell it to. This means that it is trying to install the ca-certificates-java
package from jessie, and not the backports repo, and that package is too old.

The reason you don't see that for the jdk package itself is that apparently apt thinks
that if a package is only available in the backports repo it will use it.

We can debate the merits of apt's design here, but I'm pretty sure this means
that there isn't anything that Debian will fix.

I have found the easiest solution to fix this, you can specify the target to install
the packages from with the -t switch. Therefore if you can change your
package install command for Debian jessie to

apt-get install -t jessie-backports openjdk-8-jdk-headless

it will work.

Alternatively, for jessie you can run this command

apt-get install openjdk-8-jdk-headless ca-certificates-java/jessie-backports

Thanks,

James

[bmicklea]

Thanks @james-w. The other option we discussed is to pre-install the JDK. This is probably the safest method.

@jotamartos - I think you're with Bitnami (right?) can you make one of the above changes to all the Debian-based Bitnami stacks? Right now no one can use them...

[amrecio]

@bmicklea You're right. @jotamartos, @james-w and I are with Bitnami.

We agree that the best course of action is to prevent this from happening in the future. We'll include the dependencies in our containers and send a PR to update the definition in Che.

Thanks for looking into the issue!

[riuvshin]

hey guys, I've just merged fix into master. and we will release a bugfix version next week.
but you can fix stack even now by updating them with preinstalled jdk. btw this change also will increase workspace boot time.

[bmicklea]

Reopening the issue because the stacks are still broken for the moment. We should only close when the stacks are actually fixed either by the Bitnami change or when the fix is released.

[jloramas]

Hi @riuvshin, I will prioritise working on having those dependencies preinstalled for the next week, but I cannot ensure you. Can you tell us when are you planning to do the merge next week and what is the frequency you do this bug fixing releases so we can sync with you better?

Also, do not hesitate to refer us earlier in any bug case you suspect we can help on.

[amrecio]

@riuvshin Is the list of dependencies that should be installed in Bitnami containers for Eclipse Che 5.x this one? Is there any other dependency besides the required and optional dependencies?

[riuvshin]

closing issue as it is fixed already on codenvy.io

[amrecio]

@riuvshin PR #4116 updates the Bitnami stack images to fix this issue.