-
Notifications
You must be signed in to change notification settings - Fork 706
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SDK archiving is very slow on zLinux machines #659
Comments
Personally I didn't even know this was an option.
This sounds like a great improvement since the tests usually take more than 10 minutes anyway.
This is also a good solution. |
Has anyone tried contacting the community system admins to see if this can be improved? |
We have not. I didn't necessarily want to "complain" about something that is free. I do have a contact name I can reach out to but I wouldn't mind hearing from @mstoodle first since he was the one who got the machines. |
We don't have to "complain" :) . We can just reach out to ask if that performance is unusual or "by design. Then we can decide if we want to change how we handle PR build archiving. Sound reasonable? |
Email sent to Cindy asking about the network perf |
Paraphrased summary response from Cindy back in November was
While zLInux continues to be the slowest platform (by far) to archive the SDKs, we no longer keep them in the PR builds because we simply do not have the storage space at Eclipse (#1017). Therefore this is only a problem in the OMR acceptance and the nightly. Since zLinux isn't our slowest platform overall, this is likely not a pressing issue. I will close this but we can revisit later if we feel the need. |
Currently in all Pull Request builds, once the SDK is compiled, we "archive" the sdk so it can be downloaded by anyone from the Jenkins site. This was requested via #91. The problem is that the zLinux machines we are using, appear to have a very small pipe to the outside world and it can take 10 minutes to push a ~200M sdk back to the Jenkins master host machine. The pLinux machines generally complete the archiving in under 2 minutes.
The current implementation of the pull request build archive step is serially done between the compile and test steps.
The other, more severe use case, is when a compile and test happen on different machines. The SDK has to be pushed up to the master and then pulled down to the node(s) for testing. Currently the PR builds compile/test on 1 machine, but this may not always be the case, especially if/when we have cross compiled platforms.
Is it useful to keep the sdk for a PR build and how often are they being downloaded by users? If the answer is no and almost never, then we should just stop doing it to save the time. If this is not the case then I think we should investigate archiving it in parallel with the testing (after the compile completes). I think this should be an easy fix. Having the archive run at the same time as testing I imagine wouldn't be a problem. The other thing we could think about implementing is an optional archive. We could do this by adding an extra "comment flag" in the PR Trigger Comment to indicate you would like the sdk to be archived. By default the flag would be off.
The text was updated successfully, but these errors were encountered: