No response from API when serving LightGBM model

[demirev]

I am trying to train a LightGBM, and then serve it using RestRServe. However after making a request to the microservice the process hangs and I receive no response. I also don't receive any error or warning messages.

Running top reveals that the forked process has been created and some memory has been allocated (In the actual workflow where I encountered the issue CPU usage by the forked process initially increases then drops to 0. In the example below CPU usage is negligible, so I couldn't track it).

Here is a minimal reproducible example:

library(lightgbm)
library(RestRserve)

data(agaricus.train, package = "lightgbm")
data(agaricus.test, package = "lightgbm")
train <- agaricus.train
test <- agaricus.test
bst <- lightgbm(
  data = train$data,
  label = train$label,
  num_leaves = 4,
  learning_rate = 1,
  nrounds = 2,
  objective = "binary"
)


dummy_api_function  <- function(request, response) {
  result <- predict(bst, test$data)[1]
  response$body <- jsonlite::toJSON(result)
  response$content_type <- "application/json"
  response$headers <- character(0)
  response$status_code <- 200L
  forward()
}

RestRserveApp <- RestRserve::RestRserveApplication$new()
RestRserveApp$add_post(path = "/api/dummy_api", FUN = dummy_api_function)
RestRserveApp$run(8001)

And an example of a curl request that leads to the process hanging:

curl --header "Content-Type: application/json" --request POST --data '{"foo":"bar"}' localhost:8001/api/dummy_api

(The same example but replacing result <- predict(bst, test$data)[1] with result <- 1 produces a result, so the issue must be with the predict call).

It is hard to tell whether the issue relates more to RestRserve or to LightGBM, but any help tracking the cause is appreciated.

Environment info:

> sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=bg_BG.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=bg_BG.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=bg_BG.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=bg_BG.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] Matrix_1.2-12       RestRserve_0.1.0.13 lightgbm_2.2.4      R6_2.4.0           

loaded via a namespace (and not attached):
[1] compiler_3.4.4    magrittr_1.5      tools_3.4.4       yaml_2.2.0        Rserve_1.7-3      grid_3.4.4        data.table_1.12.0 jsonlite_1.6     
[9] lattice_0.20-35

[artemklevtsov]

Seem it's LightGBM issue related with lightgbm:::Predictor$public_methods$predict which called from the lightgbm:::Booster$public_methods$predict.
Same here with lgb.save and lgb.load in the dummy_api_function.

[deann88]

I have stumbled upon the same issue, and it actually is a very big problem for R in general, as if you are developing models for production, lgb is one of the most useful libraries out there, and if you can not deploy it with Rserve, what else could one be using? Python!?
I believe RestRserve is a great contribution to the R ecosystem and the R community should pay more attention to productionizing models using R.

[dselivanov]

I haven't had a chance to check your example yet. May be there is some issue with forking - I believe lgbm use external pointers to internal C/C++ data structures. Have you tried to deploy it with RestRserve in "cooperative" mode (this doesn't make fork for each request)? Also I suggest to update Rserve to the latest version - 1.8.6 - install.packages('Rserve',,'http://www.rforge.net/')

…

On Fri, Jun 7, 2019 at 8:58 PM Deyan Spasov ***@***.***> wrote: I have stumbled upon the same issue, and it actually is a very big problem for R in general, as if you are developing models for production, lgb is one of the most useful libraries out there, and if you can not deploy it with Rserve, what else could one be using? Python!? I believe RestRserve is a great contribution to the R ecosystem and the R community should pay more attention to productionizing models using R. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#28>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABHC5XJLSRJ2UXMYOMLEVVDPZKHU5ANCNFSM4HTAUIFA> .

-- Regards Dmitriy Selivanov

[artemklevtsov]

@dselivanov I confirm this issue with the Rserve 1.8.6. bst$predict hungs anyway even if dump the model load in the dummy_api_function.

[artemklevtsov]

Cross posted here: microsoft/LightGBM#2217

[dselivanov]

I'm closing this int favor of the microsoft/LightGBM#2217 - let's keep track of the issue there.