using LIME with caret and method="nnet"

[sfd99]

Lime R-pkg is great!
Thanks to your suggestions,
the code now works fine.

2 quick Questions

Q1:
Is there a way to make the
plot_features(explanation[1:8, ])
display the plot cases (15,18,25,7) in the same order
as the cases in the test.set (7,15,18,25...) ?

reason: it's easier to present results to the end User
if the case numbers are in the same order
as in the test.set file...

Q2:
Would it be possible
to include inside/next to each color plot bar,
the actual value of a column ?
(ie: next to condition:
Petal.Length <= 1.6
you would display the actual value for Case 7: 1.4)

reason: avoids User having to consult the test.set file for each Case in plot .
The tested value is right there, in the plot... :-)

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Complete example code below :
inTrain <- createDataPartition(y=iris$Species, p=0.75, list=FALSE) # 75% for the train.set
train.set <- iris[inTrain,]
test.set <- iris[-inTrain,]

model <- train(Species ~ ., train.set, method='nnet', trace = FALSE, preProc = c("center", "scale"))
prediction <- predict(model, test.set[-5])
table(prediction, test.set$Species)
prediction <- predict(model, test.set[-5], type="prob")

now LIME!
Create an explainer object
explainer <- lime(train.set, model)

Explain new observation:
explanation <- explain(test.set[,-5], explainer, n_labels = 1, n_features = 2)
plot_features(explanation[1:8, ])

head(test.set)
Sepal.Length Sepal.Width Petal.Length
7 4.6 3.4 1.4
15 5.8 4.0 1.2
18 5.1 3.5 1.4
25 4.8 3.4 1.9
28 5.2 3.5 1.5
29 5.2 3.4 1.4

Thanks Thomas!!

[thomasp85]

Posting the original mail for reference:

Hi Thomas,

The LIME R-pkg is truly great!

Your iris example
and color plot w/method="rf",
works 100%. 👍

Q:
When I try lime with iris,
using caret w/ Neural Net method="nnet",
the plot_features() does not show any bars at all...

Here's the caret & lime code I tried:

inTrain <- createDataPartition(y=iris$Species, p=0.75, list=FALSE)  # 75% for the trainset 
  train.set <- iris[inTrain,]
  test.set  <- iris[-inTrain,]

model <- train(Species ~ ., train.set, method='nnet', trace = FALSE, preProc = c("center", "scale"))
prediction <- predict(model, test.set[-5], type="prob") 

# now LIME!
# Create an explainer object
explainer <- lime(train.set, model)

# Explain new observation
explanation <- explain(test.set, explainer, n_labels = 1, n_features = 2)

plot_features(explanation) 
# but the plot shows up 
# with almost no info or color bars... :-(

I'm not a top R expert
and sure it's something basic I am missing
in my example code...
Help Thomas!

[thomasp85]

These are both good suggestions.

Q1: This is an easy fix that I can do immediately
Q2: I need to think about how to implement this with bloating the visual with text and numbers

[sfd99]

Thanks, Thomas!.

As for Q2 (above),
some suggested (minimally intrusive) solutions,
for the lime plots

   Petal.Width <= 0.4  (0.7016)  greenbar...
   Petal.Width <= 0.4 : 0.7016   greenbar...
   Petal.Width <= 0.4 | 0.7016   greenbar...
where:
   0.7016.. is the actual value of Petal.Width
for the Case 1 ex.
(value rounding selectable as a param in plot call
 ... default=no rounding)

These actual Case values visible in the plot,
would be very helpful info feedback
for end Users. :-)

[thomasp85]

That is the most obvious choice - My main concern is that the text is already long as it is so I would prefer not to add a floating point number there as well...

[thomasp85]

Q1 has been fixed in 84f82eb

[sfd99]

Thanks for fixing Q1, Thomas!.

Hope you can still find
an acceptable solution for Q2,
(showing the value in the plot). = Useful!

How about showing the actual value of a variable,
when the User hovers the mouse
on a plot line?.

I don't know if that is possible,
but at least the actual values would not "clutter" the plot... :-)