Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

METRON-2143 Travis Build Fails to Download Maven #1433

Closed
wants to merge 3 commits into from

Conversation

@nickwallen
Copy link
Contributor

nickwallen commented May 24, 2019

This is a transient error that occurs occasionally when the Apache Mirrors are temporarily unavailable and the build is unable to download Maven.

0.50s$ wget https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.zip
--2019-05-22 21:24:16--  https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.zip
Resolving archive.apache.org (archive.apache.org)... 163.172.17.199
Connecting to archive.apache.org (archive.apache.org)|163.172.17.199|:443... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2019-05-22 21:24:17 ERROR 503: Service Unavailable.
The command "wget https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.zip" failed and exited with 8 during .

I changed the logic to download the file with curl so that it will be retried on a transient error like this.

Testing

I had to spin-up a Ubuntu Trusty host to try this out on. I found that the version of wget on Trusty does not support --retry-on-http-errror. To get the retry logic on Trusty we need to use curl.

$ curl --retry 10 -O https://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8415k  100 8415k    0     0  2050k      0  0:00:04  0:00:04 --:--:-- 2050k

I tested these options by using an online service that can replicate a 503 error for testing.

vagrant@node2:~$ curl --retry 5 https://httpstat.us/503
Warning: Transient problem: HTTP error Will retry in 1 seconds. 5 retries
Warning: left.
Warning: Transient problem: HTTP error Will retry in 2 seconds. 4 retries
Warning: left.
Warning: Transient problem: HTTP error Will retry in 4 seconds. 3 retries
Warning: left.
Warning: Transient problem: HTTP error Will retry in 8 seconds. 2 retries
Warning: left.
Warning: Transient problem: HTTP error Will retry in 16 seconds. 1 retries
Warning: left.
503 Service Unavailable503 Service Unavailable503 Service Unavailable503 Service Unavailable503 Service Unavailable503 Service Unavailable

Ultimately, the Travis CI builds will validate this change.

Pull Request Checklist

  • Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
  • Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
  • Has your PR been rebased against the latest commit within the target branch (typically master)?
nickwallen added 3 commits May 24, 2019
…y-on-http-error. Switching to curl for retries
@mmiklavc

This comment has been minimized.

Copy link
Contributor

mmiklavc commented May 24, 2019

I'm +1 by inspection pending Travis. I also like @justinleet's suggestion of caching Maven - not sure we need to download it every time.

@nickwallen

This comment has been minimized.

Copy link
Contributor Author

nickwallen commented May 24, 2019

Yes, I'd be all for caching whatever we can; including Maven.

@asfgit asfgit closed this in ccb5bec May 26, 2019
@nickwallen nickwallen mentioned this pull request Jun 24, 2019
3 of 3 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.