Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: hide maven download text to prevent log overflow #6919

Closed
rnveach opened this issue Jul 24, 2019 · 9 comments

Comments

@rnveach
Copy link
Member

commented Jul 24, 2019

PR: #6918 (comment)
Example: https://travis-ci.org/checkstyle/checkstyle/jobs/562953991

this The job exceeded the maximum log length, and has been terminated.

When the log overflows, it terminates the CI processing and marks as a red in github. We can't determine if it is a true failure or not because the downloads take so much log space that we don't know the results of the CI.

We should try to find a way to prevent this log overflow.

https://stackoverflow.com/questions/21638697/disable-maven-download-progress-indication
Some people are saying to use batch mode but its questionable how well it works.

@strkkk

This comment has been minimized.

Copy link
Collaborator

commented Jul 24, 2019

in same SO thread someone recommends to disable bad logger, looks like an option.
https://stackoverflow.com/a/35653426

@romani

This comment has been minimized.

Copy link
Member

commented Jul 24, 2019

Even further in same link , https://stackoverflow.com/a/55781461

@rnveach

This comment has been minimized.

Copy link
Member Author

commented Jul 26, 2019

Even further in same link , https://stackoverflow.com/a/55781461

Please note:

Starting with Maven 3.6.1

Taken from https://travis-ci.org/checkstyle/checkstyle/jobs/562953991#L129

mvn version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z)

So --no-transfer-progress doesn't seem like an option yet, but it could be eventually.
There is always the option to manual install a new version. See travis-ci/enterprise-installation#14 (comment) .

batch mode

This didn't work on my local (version 3.3.9) so I doubt it will work on travis.

https://stackoverflow.com/a/35653426

might be the only other option.

@romani

This comment has been minimized.

Copy link
Member

commented Jul 26, 2019

If we use travis-ci/enterprise-installation#14 (comment) , it will be good.

@rnveach

This comment has been minimized.

Copy link
Member Author

commented Jul 26, 2019

You are still ok with it @romani if it forces us to locally install the new maven too just to be able to run the scripts?

@romani

This comment has been minimized.

Copy link
Member

commented Jul 27, 2019

it will not force us to upgrade, as we can continue to run "mvn verify" on our local. This update will affect only Travis script, so it will affect you only if run validation from travis commands. I presume such travis command is executed only by Travis and some new contributors who see a failure in CI (in this case user do not care and ok with extra download).

@romani

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

in travis log file - https://api.travis-ci.org/v3/job/565950801/log.txt
looks like travis grep such output from to make web log - https://travis-ci.org/checkstyle/checkstyle/jobs/565950801

Downloading from google-maven-central: https://maven-central.storage-download.googleapis.com/repos/central/data/com/fasterxml/oss-parent/11/oss-parent-11.pom
Progress (1): 0.6/23 kB
Progress (1): 2.0/23 kB
Progress (1): 3.4/23 kB
Progress (1): 4.8/23 kB
Progress (1): 6.1/23 kB
Progress (1): 7.5/23 kB
Progress (1): 8.2/23 kB
Progress (1): 9.6/23 kB
Progress (1): 11/23 kB 
Progress (1): 12/23 kB
Progress (1): 14/23 kB
Progress (1): 15/23 kB
Progress (1): 16/23 kB
Progress (1): 18/23 kB
Progress (1): 19/23 kB
Progress (1): 21/23 kB
Progress (1): 22/23 kB
Progress (1): 23 kB   

I tried batch mode to see if it will suppress Progress .....
It works fine, but downloading takes about 14 minutes for install phase, so performance hit is enormous if maven cache is not found. Phase site require even more artifacts, so it is extra 10 minutes to download, but "site" is used only in two build items, so lets use only install.
Before such warm up, time of build item was about 3 minutes, now become about 17. But I hope that loss of cache will not be frequent.
BUT bad performance is better than failed state of build.

romani added a commit that referenced this issue Aug 1, 2019
@romani

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

fix is merged.

@romani romani closed this Aug 1, 2019

@romani romani added this to the 8.24 milestone Aug 1, 2019

romani added a commit to checkstyle/contribution that referenced this issue Aug 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.