Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set booster param nthread only at first model loading. #90

Merged
merged 1 commit into from
Feb 5, 2020

Conversation

ericangelokim
Copy link
Contributor

@ericangelokim ericangelokim commented Feb 5, 2020

Description of changes:

The method load_model is called every invocation to get the cls.format of the booster. However, cls.booster.set_param('nthread', 1) was reset every time the method was called, and in XGB 0.90 this caused a latency difference in model loading of almost two magnitudes.

# Current
#       ContentTypeParsing  DataParsing  ModelLoading  Preprediction   Prediction  PostPrediction  TotalRuntimes
#count         5000.000000  5000.000000   5000.000000    5000.000000  5000.000000     5000.000000    5000.000000
#mean             0.000232     0.001062      0.000614       0.000016     0.000095        0.000026       0.002044
#std              0.000015     0.000086      0.000155       0.000001     0.000032        0.000003       0.000247
#min              0.000219     0.001000      0.000592       0.000015     0.000087        0.000024       0.001950
#50%              0.000227     0.001046      0.000601       0.000016     0.000090        0.000025       0.002010
#75%              0.000231     0.001074      0.000608       0.000016     0.000093        0.000026       0.002049
#90%              0.000250     0.001118      0.000643       0.000017     0.000106        0.000027       0.002137
#99%              0.000274     0.001264      0.000699       0.000018     0.000131        0.000036       0.002392
#max              0.000405     0.002798      0.011242       0.000049     0.001668        0.000071       0.016163

# Pushing setting nthreads = 1
#       ContentTypeParsing  DataParsing  ModelLoading  Preprediction   Prediction  PostPrediction  TotalRuntimes
#count         5000.000000  5000.000000   5000.000000    5000.000000  5000.000000     5000.000000    5000.000000
#mean             0.000228     0.001069      0.000008       0.000014     0.000089        0.000026       0.001435
#std              0.000016     0.000090      0.000063       0.000001     0.000028        0.000002       0.000175
#min              0.000216     0.001004      0.000004       0.000012     0.000081        0.000024       0.001348
#50%              0.000223     0.001054      0.000005       0.000014     0.000084        0.000026       0.001409
#75%              0.000227     0.001082      0.000006       0.000014     0.000087        0.000027       0.001444
#90%              0.000246     0.001119      0.000006       0.000015     0.000100        0.000027       0.001492
#99%              0.000271     0.001264      0.000007       0.000019     0.000128        0.000034       0.001707
#max              0.000465     0.002782      0.001994       0.000055     0.000908        0.000080       0.006040

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Contributor

@aws-patlin aws-patlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work finding this bug!

@ericangelokim ericangelokim merged commit a7b1af6 into aws:master Feb 5, 2020
@ericangelokim ericangelokim deleted the model_loading branch February 5, 2020 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants