-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lack of cache increases Kokoro flakiness #3284
Comments
|
https://kokoro.corp.google.com/job/grpc/job/java/job/v1.4.x/job/windows/job/release/job/windows/7/
|
|
|
|
|
Would switching to a google operated mirror help a little with the connection flakiness? |
If the mirror is near GCP, maybe. To me the problem is assuming that networks and services are 100% reliable. But maybe swapping the mirror improves the situation even if it doesn't fix the problem. |
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/macos/job/continuous/job/macos/31/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/macos/job/continuous/job/macos/57/ Multiple errors in the form:
Although these are obviously misconfigurations of maven.org, this type of failure would have been less likely if we weren't doing I/O during the build. |
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/macos/job/presubmit/job/macos/626/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/macos/16/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/cronet/23/
|
This avoids needing to build all of grpc as well as avoids having to manually track which dependencies are necessary. This cuts 1.5 minutes off the Cronet build time (to 3.5 minutes). But it also reduces the amount being downloaded which should help with grpc#3284.
This avoids needing to build all of grpc as well as avoids having to manually track which dependencies are necessary. This cuts 1.5 minutes off the Cronet build time (to 3.5 minutes). But it also reduces the amount being downloaded which should help with #3284.
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/windows/34/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/presubmit/job/windows/146/ with same failure to create TLS connection. |
https://kokoro2.corp.google.com/job/grpc/job/java/job/v1.10.x/job/macos/4/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/presubmit/job/gae-interop/336/
|
https://kokoro2.corp.google.com/job/grpc/job/java/job/master/job/macos/58/
|
A band aid for #3284, to make its symptoms less noticeable.
https://source.cloud.google.com/results/invocations/57be0f6a-25fd-4430-a3e7-5faf5f98f008 (mac build)
|
https://source.cloud.google.com/results/invocations/c0ea6023-4290-4133-b5d8-11debc10f6cd (cronet build)
|
https://source.cloud.google.com/results/invocations/c84f39e4-7d17-4877-89bf-21e187d6d39c (mac build)
|
https://source.cloud.google.com/results/invocations/acc78bce-b647-4f2f-9e35-730f2d8fe695 (mac build)
|
https://source.cloud.google.com/results/invocations/994a492e-c859-4bbe-bd60-a1de19acb5ca/log (macos)
|
https://source.cloud.google.com/results/invocations/b795a23a-6f65-4888-baba-086da2f07a12 (linux_artifacts)
|
https://source.cloud.google.com/results/invocations/03d34bbc-e134-4842-b32f-2b146bf4373a/log (android)
|
https://source.cloud.google.com/results/invocations/af0dfdbe-08fb-4072-9633-6ebfddd819ea (linux_artifacts)
|
https://source.cloud.google.com/results/invocations/3da443ea-c831-4919-a613-d2935dbcdb9b (windows)
|
https://source.cloud.google.com/results/invocations/3fd5e7fd-1d1d-44b9-a663-b51d1d48b2a3 (windows)
|
https://source.cloud.google.com/results/invocations/bba86ab0-db34-467d-a1f0-35664a06ea95/log (gae-interop)
|
https://source.cloud.google.com/results/invocations/ea569152-1261-4f87-8927-228eb020987f/log (android-interop)
|
https://source.cloud.google.com/results/invocations/61aee677-d93f-4600-8952-9dc7ba5fef36/log (linux_artifacts)
|
https://source.cloud.google.com/results/invocations/a8e081ab-65d3-4eba-9099-3bdb90659e99/log (android)
|
https://source.cloud.google.com/results/invocations/3de4de08-a96a-4943-828b-7a08f0c84a45/log (windows)
|
https://source.cloud.google.com/results/invocations/557925a5-4fb8-4b45-b07e-86d68bf67279 (linux)
|
https://source.cloud.google.com/results/invocations/5af46748-c3bf-4de7-9afc-544397edac9c (linux)
|
:( I thought that the Bad Gateway from dl.google.com may have stopped. But apparently it is still happening. https://source.cloud.google.com/results/invocations/c08bd3a3-7937-4b71-992a-ae48ab362f53/log (android)
|
https://source.cloud.google.com/results/invocations/462d0686-9e6f-437e-8246-777b0d16dc7a (macos)
|
https://source.cloud.google.com/results/invocations/034e5a08-24b6-463b-8d55-16a40b86230a (android)
|
https://source.cloud.google.com/results/invocations/ddaa1955-0135-40a9-b103-f84d4b3342e6/log (linux_artifacts)
|
https://source.cloud.google.com/results/invocations/6c9ea1fc-23ba-4815-8f9f-1dfd2f4efa73 (gae interops)
|
https://source.cloud.google.com/results/invocations/b34e539b-6ba9-4bd7-a5b9-785fcd739ce1/log (gae-interop)
https://source.cloud.google.com/results/invocations/a1277d0b-4278-4907-b20e-29463d10b61b/log (android)
|
https://source.cloud.google.com/results/invocations/32df693b-09cd-4410-918e-6bfc81635f97/log (gae-interop)
|
https://source.cloud.google.com/results/invocations/89516afb-39e3-4a46-aaac-77f9148194c1 (linux)
|
https://source.cloud.google.com/results/invocations/9a62a7ac-e4d8-42e1-97ca-da8177b110f6 (linux)
|
https://source.cloud.google.com/results/invocations/5c14edae-7d01-4102-95b0-bced11c9668b (linux)
|
https://source.cloud.google.com/results/invocations/251e92e1-0541-4095-9905-21a967495dd9/targets (linux_artifacts)
|
https://source.cloud.google.com/results/invocations/75459a7d-ea75-43e2-a30e-c293bc3a08bd/log (windows)
|
https://source.cloud.google.com/results/invocations/be888841-1aa8-4480-bd8f-1751c1144c37/log (gae-interop)
|
https://source.cloud.google.com/results/invocations/cc808e82-fdd6-4392-a7b0-620743d5acdb/log (windows)
|
Gradle 5.0 implements retry for download failures. It may help address this issue. |
This is a grab-bag for any time Kokoro fails during downloading resources or similar. We cache dependency downloads on our other CIs, so such problems should be more rare. This issue should only be for flakes where caches on Travis CI or Jenkins would have been able to circumvent the failure.
The text was updated successfully, but these errors were encountered: