refactor: change the Tasks enum, remove metric operator (move the fun…

…ctionnality in the metric instead), fix regression_operator, update the tests and enhance the documentation

docs/api/attributions/operator.md

@@ -7,7 +7,7 @@
 
 ## Leitmotiv
 
-The `operator` parameter was introduced to offer users a flexible way to adapt current attribution methods or metrics. It should help them to empirically tackle new use-cases/new tasks. Broadly speaking, it should amplify the user's ability to experiment. However, this also imply that it is the user responsability to make sure that its derivationns are in-scope of the original method and make sense.  
+The `operator` parameter was introduced to offer users a flexible way to adapt current attribution methods or metrics. It should help them to empirically tackle new use-cases/new tasks. Broadly speaking, it should amplify the user's ability to experiment. However, this also imply that it is the user responsability to make sure that its derivations are in-scope of the original method and make sense.
 
 ## Operators' Signature
 
@@ -18,46 +18,10 @@ An `operator` is a function $g$ that we want to explain. This function take as i
 - `targets`: One of the following: a `tf.Tensor` or a `np.ndarray`
 
 !!!info
-    More specification concerning `model` or `inputs` can be found in the [model's documentation](../model/)
+    More specification concerning `model` or `inputs` can be found in the [model's documentation](../model/). More information on `targets` can be found [here](#tasks) or also in the [model's documentation](../model/#tasks)
 
 This function $g$ should return a **vector of scalar value** of size $(N,)$ where $N$ is the number of input in `inputs` -- *i.e* a scalar score per input.
 
-## Providing custom operator
-
-If you provide a custom operator you should be aware that:
-
-- An assertion will be made to ensure it respects the signature describe in the previous section
-- Your operator will go through the `get_gradient_of_operator` method if you use any white-box explainer
-
-```python
-def get_gradient_of_operator(operator):
-    """
-    Get the gradient of an operator.
-
-    Parameters
-    ----------
-    operator
-        Operator to compute the gradient of.
-
-    Returns
-    -------
-    gradient
-        Gradient of the operator.
-    """
-    @tf.function
-    def gradient(model, inputs, targets):
-        with tf.GradientTape() as tape:
-            tape.watch(inputs)
-            scores = operator(model, inputs, targets)
-
-        return tape.gradient(scores, inputs)
-
-    return gradient
-```
-
-!!!tip
-    Writing your operator with only tensorflow functions should increase your chance that this method does not yield any errors.
-
 ## How is the operator used in Xplique ?
 
 ### Black-box attribution methods
@@ -69,7 +33,7 @@ More concretely, for this kind of approach you want to compare some valued funct
 ```python
 original_scores = operator(model, original_inputs, original_targets)
 
-# depending on the attribution method this `perturbation_function is different`
+# depending on the attribution method this `perturbation_function` is different
 perturbed_inputs, perturbed_targets = perturbation_function(original_inputs, original_targets)
 perturbed_scores = operator(model, perturbed_inputs, perturbed_targets)
 
@@ -79,12 +43,10 @@ diff_scores = math.sqrt((original_scores - perturbed_scores)**2)
 
 ### White-box attribution methods
 
-Those methods usually require some gradients computation. The gradients that will be used are the one of the operator function (see the `get_gradient_of_operator` method in the previous section). 
+Those methods usually require some gradients computation. The gradients that will be used are the one of the operator function (see the `get_gradient_of_operator` method in the [Providing custom operator](#providing-custom-operator) section). 
 
 ## Default Behavior
 
-### Attribution methods
-
 A lot of attribution methods are initially intended for classification tasks. Thus, the default operator `predictions_operator` assume such a setting
 
 ```python
@@ -118,18 +80,26 @@ That is a setting where the variable `model(inputs)` is a vector of size $(N, C)
 !!!info
     Explaining the logits is to explain the class, while explaining the softmax is to explain why this class is more likely. Thus, it is recommended to explain the logit and exclude the softmax layer if any.
 
-### Metrics
+## Existing operators and how to use them
+
+At present, there are at present 2 operators available (and 2 others should be released soon) in the library that tackle different tasks.
+
+### Tasks
 
-It is recommended when one initialize a metric to use the same `operator` than the one used for the attribution methods. **HOWEVER** it should be pointed out that the default behavior **add a softmax** as faithfulness metrics measure a "drop in probability". Indeed, as it is better to look at attributions for models that "dropped" the final softmax layer, it is assumed that it should be added when using metrics object.
+#### Classification Tasks
+
+!!!tip
+    In general, if you are doing classification tasks it is better to not include the final softmax layer in your model but to work with logits instead!
+
+For classification tasks, it is expected for the user to use the `predictions_operator` when initializing an explainer:
 
 ```python
-def classif_metrics_operator(model: Callable,
-                             inputs: tf.Tensor,
-                             targets: tf.Tensor) -> tf.Tensor:
+@tf.function
+def predictions_operator(model: Callable,
+                         inputs: tf.Tensor,
+                         targets: tf.Tensor) -> tf.Tensor:
     """
-    Compute predictions scores, only for the label class, for a batch of samples. However, this time
-    softmax or sigmoid are needed to correctly compute metrics this time while it was remove to
-    compute attributions values so we add it here.
+    Compute predictions scores, only for the label class, for a batch of samples.
 
     Parameters
     ----------
@@ -143,48 +113,142 @@ def classif_metrics_operator(model: Callable,
     Returns
     -------
     scores
-        Probability scores computed, only for the label class.
+        Predictions scores computed, only for the label class.
     """
-    scores = tf.reduce_sum(tf.nn.softmax(model(inputs)) * targets, axis=-1)
+    scores = tf.reduce_sum(model(inputs) * targets, axis=-1)
     return scores
 ```
 
-!!!warning
-    For classification tasks, you should remove the final softmax layer of your model if you did not do it when computing attribution scores as a softmax will be apply after the call of the model on the inputs!
+- `model(inputs)`
+Consequently, we expect `model(inputs)` to yield a $(N, C)$ tensor or array where $N$ is the number of input samples and $C$ is the number of classes. 
+
+- `targets`
+If you use the default operator for classification task we expect `targets` to be a $(N, C)$ tensor or array which is a one-hot encoding of **the class you want to explain** where $N$ is the number of input samples and $C$ is the number of classes.
+
+#### Regression Tasks
+
+If the task at end is regression, then the user should instantiate the explainer with the `regression_operator`:
+
+```python
+@tf.function
+def regression_operator(model: Callable,
+                        inputs: tf.Tensor,
+                        targets: tf.Tensor) -> tf.Tensor:
+    """
+    Compute the the mean absolute error between model prediction and the target.
+    Target should the model prediction on non-perturbed input.
+    This operator can be used to compute attributions for all outputs of a regression model.
+
+    Parameters
+    ----------
+    model
+        Model used for computing predictions.
+    inputs
+        Input samples to be explained.
+    targets
+        Model prediction on non-perturbed inputs.
+
+    Returns
+    -------
+    scores
+        MAE between model prediction and targets.
+    """
+    scores = tf.reduce_mean(tf.abs(model(inputs) - targets), axis=-1)
+    return scores
+```
+
+- `model(inputs)`:
+Consequently, we expect `model(inputs)` to yield a $(N, D)$ tensor or array where $N$ is the number of input samples and $D$ is the number of variables the model should predict (possibly one). 
+
+- `targets`:
+If you are using the `regression_operator`, it is expected that `targets` to be a $(N, D)$ tensor or array of the expected multi-variate output where $N$ is the number of input samples and $D$ is the number of variables (possibly one).
 
-### Existing operators and how to use them
+#### Object-Detection Tasks
 
-At present, there are at present 4 (+1) operators available in the library:
+**Work In Progress**
 
-- The `predictions_operator` (name='CLASSIFICATION') which is the default operator and the one designed for classification tasks
-- (The `classif_metrics_operator` (name='CLASSIFICATION') which is the operator for classification tasks for metrics object)
-- The `regression_operator` (name='REGRESSION') which compute the the mean absolute error between model's prediction and the target. Target should be the model prediction on non-perturbed input. This operator can be used to compute attributions for all outputs of a regression model.
-- The `binary_segmentation_operator` (name='BINARY_SEGMENTATION') which is an operator thought for binary segmentation tasks with images. **More details are to come**
-- The `segmentation_operator` (name='SEGMENTATION') which is an operator thought for segmentation tasks with images. **More details are to come**
+#### Segmentation Tasks
 
-You can build attribution methods with those operator in two ways:
+**Work In Progress**
+
+### How to use them with an explainer ?
+
+You can build attribution methods with those operator in three ways:
 
 - Explicitly importing them
 
 ```python
 from xplique.attributions import Saliency
-from xplique.metrics import Deletion
-from xplique.commons.operators import binary_segmentation_operator
+from xplique.commons.operators import regression_operator
 
-explainer = Saliency(model, operator=binary_segmentation_operator)
+explainer = Saliency(model, operator=regression_operator)
 explanations = explainer(inputs, targets)
 ```
 
+At present, the available operators are: `predictions_operator` and `regression_operator`
+
 - Use their name
 
 ```python
 from xplique.attributions import Saliency
-from xplique.metrics import Deletion
 
-explainer = Saliency(model, operator='BINARY_SEGMENTATION')
+explainer = Saliency(model, operator='regression')
 explanations = explainer(inputs, targets)
 ```
 
+At present you can select a name in ["classification", "regression"]
+
+- Use the `Tasks` enumeration
+
+```python
+from xplique.commons import Tasks
+from xplique.attributions import Saliency
+
+explainer = Saliency(model, operator=Tasks.REGRESSION)
+explanations = explainer(inputs, targets)
+```
+
+At present the `Tasks` enum has two members: `CLASSIFICATION` and `REGRESSION`
+
+## Providing custom operator
+
+If you provide a custom operator you should be aware that:
+
+- An assertion will be made to ensure it respects the signature describe in the previous section
+- Your operator will go through the `get_gradient_of_operator` method if you use any white-box explainer
+
+```python
+def get_gradient_of_operator(operator):
+    """
+    Get the gradient of an operator.
+
+    Parameters
+    ----------
+    operator
+        Operator to compute the gradient of.
+
+    Returns
+    -------
+    gradient
+        Gradient of the operator.
+    """
+    @tf.function
+    def gradient(model, inputs, targets):
+        with tf.GradientTape() as tape:
+            tape.watch(inputs)
+            scores = operator(model, inputs, targets)
+
+        return tape.gradient(scores, inputs)
+
+    return gradient
+```
+
+!!!tip
+    Writing your operator with only tensorflow functions should increase your chance that this method does not yield any errors.
+
+!!!warning
+    Note that depending on your operator, the targets you provide should make sense
+
 ## Examples of applications
 
 **WIP**

tests/commons/test_operators.py

@@ -5,13 +5,14 @@
 import numpy as np
 import tensorflow as tf
 
+import pytest
+
 from xplique.attributions import (Saliency, GradientInput, IntegratedGradients, SmoothGrad, VarGrad,
                                   SquareGrad, GradCAM, Occlusion, Rise, GuidedBackprop, DeconvNet,
                                   GradCAMPP, Lime, KernelShap, SobolAttributionMethod,
                                   HsicAttributionMethod)
 from xplique.commons.operators import (check_operator, predictions_operator, regression_operator,
-                                       binary_segmentation_operator, segmentation_operator,
-                                       classif_metrics_operator)
+                                       binary_segmentation_operator, segmentation_operator)
 from xplique.commons.operators import Tasks, get_operator
 from xplique.commons.exceptions import InvalidOperatorException
 from ..utils import generate_data, generate_regression_model
@@ -71,34 +72,24 @@ def test_check_operator():
 def test_proposed_operators():
     # ensure all proposed operators are operators
     for operator in [predictions_operator, regression_operator,
-                     binary_segmentation_operator, segmentation_operator,
-                     classif_metrics_operator]:
+                     binary_segmentation_operator, segmentation_operator]:
         check_operator(operator)
 
 
 def test_get_operator():
     tasks_name = [task.name for task in Tasks]
-    assert tasks_name.sort() == ['CLASSIFICATION', 'REGRESSION', 'REGRESSION', \
-                                 'BINARY_SEGMENTATION', 'SEGMENTATION'].sort()
+    assert tasks_name.sort() == ['classification', 'regression'].sort()
     # get by enum
     assert get_operator(Tasks.CLASSIFICATION) is predictions_operator
-    assert get_operator(Tasks.CLASSIFICATION, is_for_metric=True) is classif_metrics_operator
     assert get_operator(Tasks.REGRESSION) is regression_operator
-    assert get_operator(Tasks.BINARY_SEGMENTATION) is binary_segmentation_operator
-    assert get_operator(Tasks.SEGMENTATION) is segmentation_operator
 
     # get by string
-    assert get_operator("classif") is predictions_operator
-    assert get_operator("Classif", is_for_metric=True) is classif_metrics_operator
-    assert get_operator("reg") is regression_operator
-    assert get_operator("bInary_Seg") is binary_segmentation_operator
-    assert get_operator("segmentation") is segmentation_operator
+    assert get_operator("classification") is predictions_operator
+    assert get_operator("regression") is regression_operator
 
     # assert a not valid string does not work
-    try:
+    with pytest.raises(AssertionError):
         get_operator("random")
-    except ValueError:
-        pass
 
     # operator must have at least 3 arguments
     function_with_2_arguments = lambda x,y: 0