New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[android-toolchain] less http in parallel #2491

Merged
merged 1 commit into from Dec 6, 2018

Conversation

Projects
None yet
2 participants
@jonathanpeppers
Contributor

jonathanpeppers commented Dec 5, 2018

Context: http://build.devdiv.io/2255858

On VSTS, I've been occasionally seeing:

android-toolchain.targets(47,5): Error : Unable to download URL `https://dl.google.com/android/repository/android-ndk-r14b-windows-x86_64.zip` to `C:\Users\dlab14\android-archives\android-ndk-r14b-windows-x86_64.zip`: Exception of type 'System.OutOfMemoryException' was thrown.
android-toolchain.targets(47,5): Error : Exception of type 'System.OutOfMemoryException' was thrown.

This is likely one of the larger files we download.

But I think we have correct usage of HttpClient here:

using (var r = await client.GetAsync (uri, source.Token)) {
    r.EnsureSuccessStatusCode ();
    using (var s = await r.Content.ReadAsStreamAsync ())
    using (var o = File.OpenWrite (tempPath)) {
        await s.CopyToAsync (o, 4096, source.Token);
    }
}

So I believe we just have too many HTTP requests in-flight at once.

Looking at android-toolchain.targets, it looks like we were running
all the Android SDK + NDK requests in parallel.

I reworked this to run the SDK requests, then the NDK requests in a
second step.

Hopefully, this will improve the OOM we are getting.

[android-toolchain] less http in parallel
Context: http://build.devdiv.io/2255858

On VSTS, I've been occasionally seeing:

    android-toolchain.targets(47,5): Error : Unable to download URL `https://dl.google.com/android/repository/android-ndk-r14b-windows-x86_64.zip` to `C:\Users\dlab14\android-archives\android-ndk-r14b-windows-x86_64.zip`: Exception of type 'System.OutOfMemoryException' was thrown.
    android-toolchain.targets(47,5): Error : Exception of type 'System.OutOfMemoryException' was thrown.

This is likely one of the larger files we download.

But I think we have correct usage of `HttpClient` here:

    using (var r = await client.GetAsync (uri, source.Token)) {
        r.EnsureSuccessStatusCode ();
        using (var s = await r.Content.ReadAsStreamAsync ())
        using (var o = File.OpenWrite (tempPath)) {
            await s.CopyToAsync (o, 4096, source.Token);
        }
    }

So I believe we just have too many HTTP requests in-flight at once.

Looking at `android-toolchain.targets`, it looks like we were running
all the Android SDK + NDK requests in parallel.

I reworked this to run the SDK requests, then the NDK requests in a
second step.

Hopefully, this will improve the OOM we are getting.

@jonathanpeppers jonathanpeppers requested a review from jonpryor Dec 5, 2018

@jonpryor jonpryor merged commit 4a02565 into xamarin:master Dec 6, 2018

3 checks passed

Ubuntu PR Build Build finished. No test results found.
Details
license/cla All CLA requirements met.
macOS PR Release Build Build finished. 208625 tests run, 1751 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment