Object Detection & Semantic Segmentation

[AntoninPoche]

Features

Adapt code to adopt new features

New operators will be added, hence, to clarify the code, operators and the relative operations were divided between operators.py and operators_operations in 7cc91cc. Naturally, the corresponding tests were adapted in 285f7dc.

Semantic Segmentation

explainer = Method(model, operator=xplique.Tasks.SEMANTIC_SEGMENTATION)

After long debates, with several persons on the team, we agreed on what we may want to explain in semantic segmentation and the relative functions to explain. All those functions were merged into the new semantic_segmentation_operator proposed in 42dbe70 and tested in fd15ed3. It supports perturbation-based and gradient-based methods. Furthermore, the semantic segmentation tutorial is on a pytorch model, thus, it is compatible with the TorchWrapper.

Furthermore, for this operator, the targets parameter is complicated to design, therefore, a set of functions was necessary. They were placed in segmentation.py in a new utils_functions/ directory in 90f7f9b, with the corresponding tests in e0ba290, and the documentation integrated to semantic segmentation documentation page.

Note, we detected that it is possible to simplify this operator, it will be done when taking into account the remarks on this PR.

Object Detection

explainer = Method(model, operator=xplique.Tasks.OBJECT_DETECTION)

Following Semantic Segmentation, a new operator: object_detection_operator was introduced in 42dbe70 and tested in fd15ed3. It also supports perturbation-based, gradient-based methods, and the TorchWrapper as shown in the object detection tutorial.

Some functions used by the object_detection_operator were initially in the attributions/object_detector.py file and moved to the utils_functions/object_detection.py file in 90f7f9b and tested in e0ba290. It was moved because the attributions/object_detector.py file should depend on operators.py and not the opposite. Furthermore, attributions/object_detector.py will be removed in future versions, see next section.

Note that a feature was requested: "We should be able to explain the different parts of an object detection prediction (i.e. box position, proba, ad class". This feature was added and can be specified by adding either " box position", " box proba", or " box class" to the initial "object detection" value of the operator parameter.

BoundingBoxExplainer is now deprecated

The initial way to apply Xplique to object detection was to use the BoundingBoxExplainer wrapper. However, with the API update, it was natural for the object detection API to match. Furthermore, no documentation was made previously, therefore BoundingBoxExplainer is now deprecated through 37d9214. Nonetheless, the new implementation of this class is just a call to the object_detection_operator which was tested in 26ee07a.

Finally, this is the first case of deprecation in Xplique, the choice was made to use the deprecated library which adds a dependency. This choice can be discussed, but this was the only one tested that showed the indicated deprecation warning message. This was added in d39ed81.

Documentation

Merge model, operator, and API description page into one

First, the docs/ directory architecture did not match the documentation architecture, it was confusing for me, hence I modified docs/ to match the documentation architecture in f5ac35e.

As @fel-thomas highlighted in #132 remarks, the documentation was too divided, furthermore, a lot of information was redundant between those pages and they were interdependent. Hence the choice was made to merge the model, operator, and API description page into one in c7483d8.

Create task related pages

Two tasks (Object Detection and Semantic Segmentation) were introduced in the documentation, their complexity induced a specific documentation page. However, it was not consistent to have documentation pages only for those two tasks. Therefore information about Classification and regression was extracted from the common API page to create two other new task-specific pages. Finally, four task-specific were introduced to the documentation in d6a72d6.

Adapt other documentation files to changes

Finally, other documentation files were adapted to the new documentation links in 731d7ac. On those files, the choice was made to add the operator specification in all snippets of code related to attribution methods. It may seem heavy but I think it improves clarity.

Others

There was a pylint error in maco file, pylint does not like isinstance(variable, Callable) hence it was changed to hastattr(variable, "__call__") in e817170.

Then, some typo were corrected in b699ad3.

[Agustin-Picard]

Awesome work! 🔥 Documentation is quite thorough and I like the way the operators are extended to object detection and segmentation. Comments are mostly typos and ways to improve the clarity of the explanations, but it really shoudn't take long to resolve them ;)

[Agustin-Picard]

LGTM. Again, awesome work!