Craft integration

[fredericboisnard]

This PR contains the integration of CRAFT in xplique.

The 4 first commits are related to the source code, the last one is about the documentation.

Base classes (1st patch of this PR: 5bfb885)

xplique/concepts/base.py : introduces base classes that will be used by current Craft classes, and potentially future other concepts methods such as ACE & ICE. In particular it contains the following base classes:

• BaseConceptExtractor: Base class for concept extraction models. This is the base class for Craft.

Craft classes (2nd patch of the PR: 3327cac)

xplique/concepts/craft.py : this file contains the majority of the code for CRAFT. It contains the following classes:

• BaseCraft: this class contains the core of all CRAFT operations, it is framework agnostic. Some specific operations for Tensorflow or Pytorch are declided in CraftTf or CraftTorch (see below).
• BaseCraftManager: this class contains the code for the CRAFT Model manager that offers the possibility to run several CRAFT instances for multiple classes. It is framework agnostic as well, so specific operations for Tensorflow or Pytorch take place in dedicated classes CraftManagerTf and CraftManagerTorch

xplique/concepts/craft_tf.py: contains Tensorflow specific implementations: CraftTf and CraftManagerTf.

xplique/concepts/craft_torch.py: contains Pytorch specific implementations: CraftTorch and CraftManagerTorch.

Tests organization (3rd patch: a5a8055)

There are 3 main tests methods which are adapted for Tensorflow and Pytorch in these files:

• tests/concepts/test_craft_tf.py
• tests/concepts/test_craft_torch.py

Here is the list of those tests:

• test_shape: use CRAFT and to process some simple matrixes of various sizes
• test_wrong_layers: check that Tensorflow or Pytorch implementation can detect if a wrong type of model is used
• test_classifier: a more complex test, based on a small classifier (whose weights are beeing loaded during the test) and a small dataset, to verify that basic concepts are correctly extracted.

Documentation (4rth patch: 487d3bb)

The documentation is split among the following files:

• README.md
• mkdocs.yml
• docs/index.md
• docs/tutorials.md
• docs/api/concepts.md
• docs/assets/craft.jpeg

[dv-ai]

1. craft.md: I think it will be good to describe a little bit the method before starting to said what the method shall requires
2. I think craft package shall support Pytorch model thanks to the pytorch wrapper developed. I think with little effort, it can be done with craft package.
3. visualization function shall be develop too and a tutorial illustrating the usage of the method on imagenet will be very useful.
4. I will change the name craft_tf by craft

[lucashervier]

Technically it looks wonderful! I mean the code is really neat and quite clear. I added some questions and remarks on those points but I had really a few points. However, there is for now 2 major drawbacks:

• I think the tests are not appropriate, in the sense that considering the methods at stake they should be more developped
• It misses documentation for both ACE and ICE + the format of the docstring is not harmonized with the other methods.
  Anyway, it is still an impressive work and the harder is done, congrats for that 🔥

[fredericboisnard]

Rebased on master

[fredericboisnard]

I repushed because I forgot to add the links to the paper + notebook in craft.md ...

[AntoninPoche]

Amazing work to adapt Craft code for it to be ready for packaging. It is nice to have both tensorflow and pytorch versions !

I made several comments but I did not finish my review as I did not read the tutorials yet, I will do it tomorrow. Furthermore, I do not have a deep understanding of the method so my remarks are more high-level.

Finally, it seems that your code did not pass the test workflow. You should use pylint xplique and pytest commands to verify you code will pass the workflow.

[AntoninPoche]

Great job again, the plots are really pertinent!

However, the documentation is a bit lacking and I have a few remarks on the tutorials:

• When you make a command line in a notebook, make sure to add the quiet argument to reduce verbosity (-q for the pip install).
• The tutorials should be added to the Xplique Google Drive to be with the others.
• Some plot functions are called for each image separately and thus they plot the result in one column. I think it hinders fluidity.
• One thing I like from other tutorials is that we quickly get an example of results. Here I would make a cell with the fewest lines possible to obtain the all-in-one plot_image_concepts plot. Then describe the steps as you do now.
• In fact, I would add a quick general description of the method with a schema at the top as done in other unit tutorials, for example for Rise: https://colab.research.google.com/drive/1icu2b1JGfpTRa-ic8tBSXnqqfuCGW2mO.
• You do not appear as an Author, are you not the author of the tutorials? (The Author line does not correspond to the papers' authors).
• You did not put the Xpliqe logo at the top of the tutorials.

Nonetheless, those tutorials show awesome results and how easy it is to get them through the API so great work.

Feel free to come see me if you want to discuss my remarks.

[lucashervier]

It seems that the tests are failing because you used the tqdm package. 2 options for me: either you think it has a real added-value in which case you should add it to the dependencies (requirements.txt and setup.py) or you simply don't use it. Personnally, I have no opinion.

On the other hands, it also fails for craft torch because with the general test torch is not installed and I think we should keep it that way. You should add the test to the 'tests for torch' job. You can import torch in the related python files the same way as it is done for the Pytorch wrapper so that torch is not a global dependency :)

[AntoninPoche]

Amazing work, the PR is close to being merged!

[lucashervier]

Nice work team!

[AntoninPoche]

Well done!