V chguan/add icml ex nlp code #90

Frozenmad · 2019-06-08T13:34:01Z

Description

We have added the code of our ICML paper. The related files are:

interpreter.py and README.md files under utils_nlp\interpreter. The interpreter.py file is the main functional file we utilize. README.md is an instruction file on it.
explain_simple_model.ipynb and explain_BERT_model.ipynb files under scenarios\interpret_NLP_models for two scenarios on how to interpreter.py.
test_interpreter.py under tests\unit. This file contains 6 unit tests for interpreter.py (which, in my machine, cost about 2.25s to run).
example.png under utils_nlp\interpreter folder used by README.md, and regular.json under scenarios\interpret_NLP_models folder used by explain_BERT_model.ipynb. I know from other pull requests that files like these are not allowed to merge. So, can anyone help me upload these two files to somewhere? Thanks for your help in advance : )

Related Issues

Our issue is #62.

Checklist:

My code follows the code style of this project, as detailed in our contribution guidelines.
I have added tests.
I have updated the documentation accordingly (I now add README.md to utils_nlp only. What other .md files should I modify or add?).

review-notebook-app · 2019-06-08T13:34:13Z

Check out this pull request on ReviewNB: https://app.reviewnb.com/microsoft/nlp/pull/90

Visit www.reviewnb.com to know how we simplify your Jupyter Notebook workflows.

msftclas · 2019-06-08T13:34:14Z

All CLA requirements met.

saidbleik · 2019-06-10T04:34:14Z

This is great. Thanks!
You don't need to add any mode md files at this point. We will review it soon.

miguelgfierro · 2019-06-10T09:55:44Z

hey @Frozenmad this is really nice.

We'll take a look at the code soon.

In the meantime, all the images and data that we use in the repo are in blobs, see discussion here.

I uploaded the json and png to our blob: https://nlpbp.blob.core.windows.net/images/result.png and https://nlpbp.blob.core.windows.net/data/regular.json. Could you please remove these files and use the links?

miguelgfierro

this looks really good

tests/unit/test_interpreter.py

miguelgfierro · 2019-06-10T10:02:41Z

tests/unit/test_interpreter.py

+
+
+def test_train_fixed_length_interp(fixed_length_interp):
+    fixed_length_interp.optimize(iteration=10)


here is there something that is produced and that can be checked?

here is there something that is produced and that can be checked?

This line returns nothing but will change some parameters inside the instance fixed_length_interp. I'm testing this to make sure the data type (in this function) is matched when calculating. Do you have suggestions testing this function?

at the end of optimize there is this instruction self.load_state_dict(state_dict), would it be possible to test that there has been some change in the state_dict?

Good idea! Thanks for your suggestion and I'll check if the regular and ratio change (regular shouldn't change and ratio should) in this instance!

tests/unit/test_interpreter.py

utils_nlp/interpreter/Interpreter.py

utils_nlp/interpreter/README.md

miguelgfierro · 2019-06-10T10:31:25Z

scenarios/interpret_NLP_models/explain_BERT_model.ipynb

@@ -0,0 +1,206 @@
+{


several of the comments I added in the other notebook applies here. Please see other comments

Reply via ReviewNB

Thanks! I'll check other comments~

miguelgfierro · 2019-06-10T10:31:25Z

scenarios/interpret_NLP_models/explain_BERT_model.ipynb

@@ -0,0 +1,206 @@
+{


is the Bert model that has 24 layers? why are you choosing 3 instead of other layer or a group of layers?

Reply via ReviewNB

The BERT model we use is the base model in paper, which contains 12 layers only, which I will clarify in the new code. We randomly take a layer in BERT (the 3rd layer here) as an example to show how to explain a certain layer using our tools.
Note that all the layers are explainable via our tools, through similar methods given in this notebook (just change some parameters). Here, we want to show how to use instead of what's the result in this notebook because the user may need to explain the layers in their own models instead of BERT or other BERT model layers.
Do you have suggestions on how to make this clearer? I think I'll change the name of this notebook to something like how to explain a layer in a pretrained model and clarify that this notebook shows only one case. Is that proper?

scenarios/interpret_NLP_models/explain_BERT_model.ipynb

scenarios/interpret_NLP_models/explain_simple_model.ipynb

Frozenmad · 2019-07-09T05:01:46Z

Thanks for your comments! My graduation ceremony is now finished and I'm back on managing this branch now. I'll try my best to meet your requirements!

Frozenmad · 2019-07-09T05:26:18Z

hey @Frozenmad this is really nice.

We'll take a look at the code soon.

In the meantime, all the images and data that we use in the repo are in blobs, see discussion here.

I uploaded the json and png to our blob: https://nlpbp.blob.core.windows.net/images/result.png and https://nlpbp.blob.core.windows.net/data/regular.json. Could you please remove these files and use the links?

Sure! Thanks for help uploading these files!

miguelgfierro · 2019-07-12T10:41:39Z

hey @Frozenmad please let me know when you are finished with the changes so I can take a look, is now the best moment to look at the code and should I wait for more changes from your part?

Frozenmad · 2019-07-14T02:00:39Z

@miguelgfierro Ops! I'm sorry. The new code is now ready for checking! Thanks for your nice reminder~
I'll let you know next time if there are other codes needed to be refined :)

miguelgfierro · 2019-07-15T11:17:47Z

utils_nlp/interpreter/Interpreter.py

+
+class Interpreter(nn.Module):
+    """ Interpreter for interpreting one instance. The method is from
+    paper [Towards a Deep and Unified Understanding of Deep Neural


minor detail, this documentation (at some point) will be parsed by sphinx. The notation for what you want is this:

`Towards a Deep and Unified Understanding of Deep NeuralModels in NLP <http://proceedings.mlr.press/v97/guan19a/guan19a.pdf>`_

It's important to add the final _ for some reason :-0

Thanks! I'll change the style : )

utils_nlp/interpreter/Interpreter.py

miguelgfierro · 2019-07-15T11:22:08Z

tests/unit/test_interpreter.py

+
+
+def test_train_fixed_length_interp(fixed_length_interp):
+    fixed_length_interp.optimize(iteration=10)


at the end of optimize there is this instruction self.load_state_dict(state_dict), would it be possible to test that there has been some change in the state_dict?

utils_nlp/interpreter/Interpreter.py

tests/unit/test_interpreter.py

miguelgfierro

this is really good

miguelgfierro · 2019-07-15T11:34:01Z

scenarios/interpret_NLP_models/explain_saved_model.ipynb

+    "x = model.embeddings(token_tensor, segment_tensor)[0]\n",
+    "\n",
+    "# extract the Phi we need to explain, suppose the layer we are interested in is layer 3\n",
+    "def Phi(x):\n",


minor suggestion here, maybe we can add a parameter called layer to make this function general, instead of harcoding the layer 3 inside the code

miguelgfierro · 2019-07-15T11:34:23Z

scenarios/interpret_NLP_models/explain_saved_model.ipynb

+    "\n",
+    "# extract the Phi we need to explain, suppose the layer we are interested in is layer 3\n",
+    "def Phi(x):\n",
+    "    global model\n",


why is making global the model needed?

Seems that the code can still run without this line hmmmmm.
I'll remove this line 0.0. Originally, I was thinking to make the model global so that we don't need to load the model we need every time we call Phi().

miguelgfierro · 2019-07-15T11:35:31Z

scenarios/interpret_NLP_models/explain_saved_model.ipynb

+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# here, we load the regularization we already calculated for simplicity\n",


here could you just print the regularization value? out of curiosity, how did you get this value?

I'll add the print here. We get this by sampling. We first sample part of the training data of BERT, then pass them to BERT and collect the 3rd hidden state and calculate the std values of every dimension of them (the std values are the regular values).

scenarios/interpret_NLP_models/explain_simple_model.ipynb

miguelgfierro · 2019-07-15T11:46:55Z

scenarios/interpret_NLP_models/explain_saved_model.ipynb

@@ -0,0 +1,234 @@
+{


hey @Frozenmad, these two notebooks are pretty small. Do you think it make sense to merge them?

Ok, I'll try to merge them together~

Frozenmad · 2019-07-17T03:07:07Z

Hi @miguelgfierro. I've uploaded new codes here! Major changes include:

Change the test style in unit test.
Change the link style to the one you provided.
Merge the two small notebooks to one.

Thanks for all your advice : )

miguelgfierro

This is awesome! amazing stuff

Frozenmad added 2 commits June 8, 2019 12:51

add explain-NLP-model part for issue #62

f4f3591

rename files and revise README for #62

f0d6a2f

saidbleik assigned saidbleik and miguelgfierro Jun 10, 2019

miguelgfierro reviewed Jun 10, 2019

View reviewed changes

add CLA, rm files according to comments, etc.

503df31

miguelgfierro reviewed Jul 15, 2019

View reviewed changes

Frozenmad added 3 commits July 17, 2019 10:32

change unit test of interpreter

5ad5e9b

change link style of interpreter

babe101

merge two notebooks

b2ca990

miguelgfierro approved these changes Jul 17, 2019

View reviewed changes

miguelgfierro merged commit 48f916f into microsoft:staging Jul 17, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V chguan/add icml ex nlp code #90

V chguan/add icml ex nlp code #90

Frozenmad commented Jun 8, 2019 •

edited

review-notebook-app bot commented Jun 8, 2019

msftclas commented Jun 8, 2019 •

edited

saidbleik commented Jun 10, 2019

miguelgfierro commented Jun 10, 2019

miguelgfierro left a comment

miguelgfierro Jun 10, 2019

Frozenmad Jul 9, 2019

miguelgfierro Jul 15, 2019

Frozenmad Jul 16, 2019

miguelgfierro Jun 10, 2019

Frozenmad Jul 9, 2019

miguelgfierro Jun 10, 2019

Frozenmad Jul 9, 2019

Frozenmad commented Jul 9, 2019

Frozenmad commented Jul 9, 2019

miguelgfierro commented Jul 12, 2019

Frozenmad commented Jul 14, 2019

miguelgfierro Jul 15, 2019

Frozenmad Jul 16, 2019

miguelgfierro Jul 15, 2019

miguelgfierro left a comment

miguelgfierro Jul 15, 2019

Frozenmad Jul 16, 2019

miguelgfierro Jul 15, 2019

Frozenmad Jul 16, 2019

miguelgfierro Jul 15, 2019

Frozenmad Jul 16, 2019

miguelgfierro Jul 15, 2019

Frozenmad Jul 17, 2019

Frozenmad commented Jul 17, 2019

miguelgfierro left a comment



		def test_train_fixed_length_interp(fixed_length_interp):
		fixed_length_interp.optimize(iteration=10)

V chguan/add icml ex nlp code #90

V chguan/add icml ex nlp code #90

Conversation

Frozenmad commented Jun 8, 2019 • edited

Description

Related Issues

Checklist:

review-notebook-app bot commented Jun 8, 2019

msftclas commented Jun 8, 2019 • edited

saidbleik commented Jun 10, 2019

miguelgfierro commented Jun 10, 2019

miguelgfierro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frozenmad commented Jul 9, 2019

Frozenmad commented Jul 9, 2019

miguelgfierro commented Jul 12, 2019

Frozenmad commented Jul 14, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miguelgfierro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Frozenmad commented Jul 17, 2019

miguelgfierro left a comment

Choose a reason for hiding this comment

Frozenmad commented Jun 8, 2019 •

edited

msftclas commented Jun 8, 2019 •

edited