Dowload failure lacking detailed traces to identify root cause

[gberche-orange]

Buildpacks highly depend on remote binaries to be present.

My custom buildpack based on the java-buildpack frequently fails on run.pivotal.io when trying to download remote packages. The current traces seem to not include details from exceptions thrown by the Net::HTTP::Get method.

2013-09-08 18:11:06,088 INFO  (397) -
app staging logs:-----> Downloaded app package (4.9M)
Initialized empty Git repository in /tmp/buildpacks/java-buildpack.git/.git/
-----> Downloading OpenJDK 1.7.0_21 JRE from http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz (11.0s)
       Expanding JRE to .java (1.0s)
-----> Downloading Jonas 5.2.4 from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz W, [
2013-09-08T16:11:05.295650 #64]  WARN -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/util/download_cache.rb:145:in `look_aside': Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz. Looking in buildpack cache.
W, [2013-09-08T16:11:05.296308 #64]  WARN -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/util/download_cache.rb:154:in `look_aside': Buildpack cache does not contain http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz. Failing the download.
FAIL
E, [2013-09-08T16:11:05.296572 #64] ERROR -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/buildpack.rb:46:in `rescue in drive_buildpack_with_logger': Compile failed with exception #<RuntimeError: Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz>
Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz
/var/vcap/packages/dea_next/buildpacks/lib/installer.rb:16:in `compile': Buildpack compilation step failed: (RuntimeError)
    from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:76:in `block in compile_with_timeout'
    from /usr/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:75:in `compile_with_timeout'
    from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:57:in `block in stage_application'
    from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in `chdir'
    from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in `stage_application'
    from /var/vcap/packages/dea_next/buildpacks/bin/run:10:in `<main>'

see

java-buildpack/lib/java_buildpack/util/download_cache.rb

Line 154 in baf557a

raise "Unable to download from #{uri}"

I'm thinking to add #{$!} into the log message

[glyn]

I added the exception to the log message in commit 9ab3c48. Thanks for the suggestion.

On 18 Sep 2013, at 16:17, Guillaume Berche notifications@github.com wrote:

Buildpacks highly depend on remote binaries to be present.

My custom buildpack based on the java-buildpack frequently fails on run.pivotal.io when trying to download remote packages. The current traces seem to not include details from exceptions thrown by the Net::HTTP::Get method.

2013-09-08 18:11:06,088 INFO (397) -
app staging logs:-----> Downloaded app package (4.9M)
Initialized empty Git repository in /tmp/buildpacks/java-buildpack.git/.git/
-----> Downloading OpenJDK 1.7.0_21 JRE from http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz (11.0s)
Expanding JRE to .java (1.0s)
-----> Downloading Jonas 5.2.4 from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz W, [
2013-09-08T16:11:05.295650 #64] WARN -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/util/download_cache.rb:145:in look_aside': Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz. Looking in buildpack cache. W, [2013-09-08T16:11:05.296308 #64] WARN -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/util/download_cache.rb:154:inlook_aside': Buildpack cache does not contain http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz. Failing the download.
FAIL
E, [2013-09-08T16:11:05.296572 #64] ERROR -- /tmp/buildpacks/java-buildpack.git/lib/java_buildpack/buildpack.rb:46:in rescue in drive_buildpack_with_logger': Compile failed with exception #<RuntimeError: Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz> Unable to download from http://orange-buildpacks-binaries.s3-website-us-west-1.amazonaws.com/jonas/jonas-full-5.2.4-bin.tar.gz /var/vcap/packages/dea_next/buildpacks/lib/installer.rb:16:incompile': Buildpack compilation step failed: (RuntimeError)
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:76:in block in compile_with_timeout' from /usr/lib/ruby/1.9.1/timeout.rb:68:intimeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:75:in compile_with_timeout' from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:57:inblock in stage_application'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in chdir' from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:instage_application'
from /var/vcap/packages/dea_next/buildpacks/bin/run:10:in `

'
see

java-buildpack/lib/java_buildpack/util/download_cache.rb

Line 154 in baf557a

raise "Unable to download from #{uri}"

I'm thinking to add #{$!} into the log message

—
Reply to this email directly or view it on GitHub.

[gberche-orange]

Thanks a lot Glyn !

[nebhale]

@gberche-orange

Next time you see the failure and get a reasonable message, can you please close this issue out if you are satisfied? Thanks.