Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn off Background fetcher in Soci Config #110

Closed
sbuckfelder opened this issue Oct 14, 2022 · 4 comments
Closed

Turn off Background fetcher in Soci Config #110

sbuckfelder opened this issue Oct 14, 2022 · 4 comments
Assignees
Labels
performance Improves performance

Comments

@sbuckfelder
Copy link
Contributor

Investigate the impact of turning off the prefetcher.

@sbuckfelder sbuckfelder added the performance Improves performance label Oct 14, 2022
@vkuzniet
Copy link
Contributor

vkuzniet commented Oct 19, 2022

Turning off background fetch does not seem to improve things. The results with and without background fetch seem to be within the same range.
Below are the results for a few runs to showcase the facts.

Getting to ready line for drupal with background fetch enabled:

{
  "testName": "SociRundrupal",
  "numberOfTests": 10,
  "testTimes": [
    3.514366104,
    3.438325975,
    3.386830468,
    3.484538835,
    3.5999828799999998,
    3.381849567,
    3.55579802,
    3.487010156,
    3.005520713,
    3.498268507
  ],
  "stdDev": 0.1571243835514334,
  "mean": 3.4352491225,
  "min": 3.005520713,
  "pct25": 3.3843400175,
  "pct50": 3.484538835,
  "pct75": 3.5063173055,
  "pct90": 3.55579802,
  "max": 3.5999828799999998
}

Getting to ready line for drupal with background fetch disabled:

{
  "testName": "SociRundrupal",
  "numberOfTests": 10,
  "testTimes": [
    3.427106556,
    3.736782046,
    3.826452474,
    3.347792385,
    4.078158608,
    3.545986215,
    3.651143014,
    3.406338622,
    3.641549722,
    4.979371583
  ],
  "stdDev": 0.45501353945730844,
  "mean": 3.7640681225000003,
  "min": 3.347792385,
  "pct25": 3.416722589,
  "pct50": 3.641549722,
  "pct75": 3.78161726,
  "pct90": 4.078158608,
  "max": 4.979371583
}

Getting to ready line for jenkins with background fetch enabled:

{
  "testName": "SociRunjenkins",
  "numberOfTests": 10,
  "testTimes": [
    10.054356415,
    9.494009386,
    9.627321754,
    9.66908636,
    11.155647402,
    9.584445945,
    9.279093966,
    9.55490319,
    9.401226454,
    9.587650685
  ],
  "stdDev": 0.5089182780597901,
  "mean": 9.7407741557,
  "min": 9.279093966,
  "pct25": 9.447617919999999,
  "pct50": 9.584445945,
  "pct75": 9.648204057000001,
  "pct90": 10.054356415,
  "max": 11.155647402
}

Getting to ready line for jenkins with background fetch disabled:

{
  "testName": "SociRunjenkins",
  "numberOfTests": 10,
  "testTimes": [
    9.361044952,
    9.750638978,
    9.947722197000001,
    9.234274562,
    9.708358833,
    9.228201503,
    9.054713528,
    9.191678314,
    9.824592823,
    9.515478995
  ],
  "stdDev": 0.2942896685220635,
  "mean": 9.4816704685,
  "min": 9.054713528,
  "pct25": 9.2099399085,
  "pct50": 9.361044952,
  "pct75": 9.7294989055,
  "pct90": 9.824592823,
  "max": 9.947722197000001
}

@vkuzniet
Copy link
Contributor

Large images also do not seem to benefit from disabling the background fetch. Running test with Tensorflow image shows the degradation in getting to ready line with background fetch disabled.

Getting to ready line for Tensorflow image with background fetch enabled:

{
  "testName": "SociRunTensorHelloWorld",
  "numberOfTests": 10,
  "testTimes": [
    19.610846123,
    19.165775153,
    20.629963265,
    20.721168575,
    19.42073217,
    19.330365578,
    19.632302849,
    20.622923324,
    19.807134995,
    20.03266827
  ],
  "stdDev": 0.5481311674358874,
  "mean": 19.8973880302,
  "min": 19.165775153,
  "pct25": 19.375548874,
  "pct50": 19.632302849,
  "pct75": 20.327795797,
  "pct90": 20.629963265,
  "max": 20.721168575
}

Getting to ready line for Tensorflow image with background fetch disabled:

{
  "testName": "SociRunTensorHelloWorld",
  "numberOfTests": 10,
  "testTimes": [
    19.65265909,
    20.893706043,
    21.212172436,
    20.133621139,
    19.299996586,
    19.629116495,
    21.004703673,
    19.637957393,
    19.86712194,
    22.457070123
  ],
  "stdDev": 0.937789828527017,
  "mean": 20.378812491799998,
  "min": 19.299996586,
  "pct25": 19.633536944,
  "pct50": 19.86712194,
  "pct75": 20.949204858,
  "pct90": 21.212172436,
  "max": 22.457070123
}

@wmesard wmesard changed the title Turn off Prefetcher in Soci Config Turn off Background fetcher in Soci Config Nov 4, 2022
@vkuzniet vkuzniet self-assigned this Nov 7, 2022
@vkuzniet
Copy link
Contributor

vkuzniet commented Nov 9, 2022

PR: #176

@vkuzniet vkuzniet closed this as completed Nov 9, 2022
@vkuzniet
Copy link
Contributor

vkuzniet commented Nov 9, 2022

Document with the results of benchmarking: Performance effects of turning off background fetch.pdf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Improves performance
Projects
Status: Done
Development

No branches or pull requests

2 participants