New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offload downloading to a separate thread pool to fix performance issue #249

Merged
merged 1 commit into from Jun 26, 2018

Conversation

Projects
None yet
3 participants
@OlegYch
Contributor

OlegYch commented Jun 25, 2018

Fixes sbt/sbt#4224

blocking+global ec+par collections leads to a significant performance regression, offload downloading to a separate threadpool

fixes sbt/sbt#4224
blocking+global ec+par collections leads to a significant performance regression, offload downloading to a separate threadpool
@eed3si9n

Thanks for the prompt fix. LGTM

@typesafe-tools

This comment has been minimized.

typesafe-tools commented Jun 26, 2018

A validation involving this pull request is in progress...

@typesafe-tools

This comment has been minimized.

typesafe-tools commented Jun 26, 2018

The validator has checked the following projects against Scala 2.12,
tested using dbuild, projects built on top of each other.

Project Reference Commit
sbt 1.1.x sbt/sbt@21db286
zinc 1.1.x sbt/zinc@899d7bd
io 1.1.x sbt/io@1534128
librarymanagement pull/249/head a8d5db4
util 1.1.x sbt/util@d40517b
website 1.1.x

The result is: FAILED
(restart)

@OlegYch

This comment has been minimized.

Contributor

OlegYch commented Jun 26, 2018

@eed3si9n

[sbt212] [info] ! Key parser test.An unspecified project axis resolves to the current project: Falsified after 22 passed tests.
[sbt212] [info] > Labels of failing property: 
[sbt212] [info] Parsed: Right(ProjectRef(uri("m:gwDk#hrkqzq"), "nf") / lv)

seems like an unrelated regression

@eed3si9n

This comment has been minimized.

Member

eed3si9n commented Jun 26, 2018

seems like an unrelated regression

Yea. There's a known issue with slash testing.

@eed3si9n eed3si9n merged commit 70ff5cc into sbt:1.1.x Jun 26, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@eed3si9n eed3si9n removed the ready label Jun 26, 2018

@eed3si9n eed3si9n changed the title from fixes https://github.com/sbt/sbt/issues/4224 to Offload downloading to a separate thread pool to fix performance issue Oct 30, 2018

eed3si9n added a commit to eed3si9n/librarymanagement that referenced this pull request Nov 1, 2018

Use fixed thread pool
Fixes sbt#274

In sbt#249 parallel download switched to using its own thread pool.
It could potentially lead to unbounded download if nobody throttled.
This works around the issue by fixing the number of thread to 6, which is a common per-host max connection count.

@eed3si9n eed3si9n referenced this pull request Nov 1, 2018

Merged

Use fixed thread pool #278

eed3si9n added a commit to eed3si9n/librarymanagement that referenced this pull request Nov 6, 2018

Use fixed thread pool
Fixes sbt#274

In sbt#249 parallel download switched to using its own thread pool.
It could potentially lead to unbounded download if nobody throttled.
This works around the issue by fixing the number of thread to 6, which is a common per-host max connection count.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment