Regression Task Scoring

[Lswhiteh]

Hi all, I'm currently wondering if there's a way to get Saliency maps on a regression task I'm attempting to visualize. I assumed that the "InactiveScore" class would be the way to go, but I get ValueErrors as such:

  File "train.nn.PYTHON3.py", line 108, in <module>
    sal_map = viz.gen_saliency(model=model, X=xtest_untransposed, score=score)
  File "/overflow/dschridelab/users/lswhiteh/saliency/pop_gen_cnn/theta/viz.py", line 13, in gen_saliency
    return saliency(score, X)  # , smooth_samples=20, smooth_noise=0.2)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tf_keras_vis/saliency.py", line 109, in __call__
    grads = self._get_gradients(seed_inputs, scores, gradient_modifier, training,
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tf_keras_vis/saliency.py", line 131, in _get_gradients
    grads = [gradient_modifier(g) for g in grads]
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tf_keras_vis/saliency.py", line 131, in <listcomp>
    grads = [gradient_modifier(g) for g in grads]
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tf_keras_vis/saliency.py", line 30, in <lambda>
    gradient_modifier=lambda grads: K.abs(grads),
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/keras/backend.py", line 2440, in abs
    return math_ops.abs(x)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/ops/math_ops.py", line 398, in abs
    x = ops.convert_to_tensor(x, name="x")
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/profiler/trace.py", line 163, in wrapped
    return func(*args, **kwargs)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1540, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 339, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 264, in constant
    return _constant_impl(value, dtype, shape, name, verify_shape=False,
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 276, in _constant_impl
    return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 301, in _constant_eager_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/nas/longleaf/home/lswhiteh/.conda/envs/saliency/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 98, in convert_to_eager_tensor
    return ops.EagerTensor(value, ctx.device_name, dtype)
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor. ```

I should mention that I'm also trying to do this with a 1DCNN architecture, maybe that's also causing issues?

Any help would be great. Thanks

[keisen]

@Lswhiteh, I'm sorry for the lack of information.
InactiveScore is the function to inactivate the model output by multiplying by zero. (For example: #14)
tf-keras-vis doesn't have the score class for Regression task, so please refer to the following.

For purposes of brevity, I assume that the shape of your model's output is (samples, 1).
And, in Regression task, we need to consider how what you want to visualize contributes to the model output,
so suppose that you are interested in one (or more) of the following:

• Increase the output value
• Decrease the output value
• Maintain the output value at ...

(Because you've attempted to use InactiveScore, you're probably interested in Decrease the output value, right.)

The score functions you need will be one (or more) of the following:

def score_when_increase(output):
    return output[:, 0]

def score_when_decrease(output):
    return -1.0 * output[:, 0]

def score_when_maintain(output):
    # If you want to visualize the region of input that contributes to maintain the output at 0.0
    return tf.math.abs(1.0 / (output[:, 0] + tf.keras.backend.epsilon()))

For example, if you want to visualize the region of input that contributes the increase output, the usage is:

saliency_maps = saliency(score_when_increase, X)

If you have any question or any problem, please feel free to ask us.
Thanks!

[Lswhiteh]

This clarified quite a bit for me, thanks for the help!