improved LIME for text data

[kmike]

This is a work-in-progress.

TODO:

• tutorial for debugging complex text processing pipelines using LIME;
• cleanup LIME documentation, remove redundancy;
• TextExplainer should handle target_names better when there is no examples of some class in the generated dataset;
• fix test coverage

This PR also fixes #102 and #39.

[codecov-io]

Current coverage is 97.27% (diff: 98.78%)

Merging #110 into master will decrease coverage by 0.03%

@@             master       #110   diff @@
==========================================
  Files            35         37     +2   
  Lines          1894       2090   +196   
  Methods           0          0          
  Messages          0          0          
  Branches        362        390    +28   
==========================================
+ Hits           1843       2033   +190   
- Misses           24         28     +4   
- Partials         27         29     +2   

Diff Coverage	File Path
••••••••• 96%	eli5/lime/lime.py
••••••••• 97%	new eli5/lime/_vectorizer.py
••••••••• 97%	eli5/lime/utils.py
•••••••••• 100%	eli5/lime/textutils.py
•••••••••• 100%	eli5/sklearn_crfsuite/explain_weights.py
•••••••••• 100%	new eli5/sklearn/_span_analyzers.py
•••••••••• 100%	eli5/utils.py
•••••••••• 100%	eli5/sklearn/utils.py
•••••••••• 100%	eli5/_graphviz.py
•••••••••• 100%	eli5/ipython.py

Review all 27 files changed

Powered by Codecov. Last update 5f4e975...aebd959

[kmike]

//cc @lopuhin I think this is ready. Do you have any comments?

Tutorial doesn't explain all options - most notably, it doesn't use position_dependent=True flag and doesn't explain rbf_sigma parameter. I know how they work and what they intend to do, but I don't know when should one use them in practice, so it is not in the tutorial :)

Tutorial: https://github.com/TeamHG-Memex/eli5/blob/d6786c56f51d8fa485829adcf641967ff8839416/notebooks/TextExplainer.ipynb

[lopuhin]

I think it would be nice to link to the paper (https://arxiv.org/abs/1602.04938)

[kmike]

Yeah, makes sense.

[lopuhin]

Hey @kmike , I just finished reading the notebook, it looks great - I like that most important stuff is at the start, and you show how it can break.

KL divergence and score of the white-box classifier predictions of the black-box classifier seem to be important to know if explanation should be trusted - does it make sense to include them in explanation output by default?

[lopuhin]

... does it make sense to include them in explanation output by default?

Hm, but I see it's not really convenient to implement, and it's not obvious that they are needed in the explanation.

The PR looks great! I didn't know about # type: (...) -> syntax, that is much more convenient.

[kmike]

I was also thinking about adding scores to the output by default, but it was not straightforward to implement indeed. Adding a custom field just for scores looks like a bit too much; putting it to description is not enough because description is hidden by default.