Optimizing slice of variable not possible

[Hoeze]

Applying the gradient of a variable slice currently results in a NotImplemented error of tf.train.Optimizer.

The following two examples are working:

### WORKING ###
X = tf.Variable(2, dtype=tf.float32)
y = tf.constant(10, dtype="float32")
loss = y - (X*X)

variables=[X]
gradient = tf.gradients(loss, variables)
gradient = [(g, v) for g, v in zip(gradient, variables)]
train_op = tf.train.AdamOptimizer().apply_gradients(gradient)

### WORKING ###
big_X = tf.Variable([2,3,4], dtype=tf.float32)
X = big_X[0]
y = tf.constant(10, dtype="float32")
loss = y - (X*X)

train_op = train_op = tf.train.AdamOptimizer().minimize(loss)

The following example throws an error:

### NOT WORKING ###
big_X = tf.Variable([2,3,4], dtype=tf.float32)
X = big_X[0]
y = tf.constant(10, dtype="float32")
loss = y - (X*X)

variables=[X]
gradient = tf.gradients(loss, variables)
gradient = [(g, v) for g, v in zip(gradient, variables)]
train_op = tf.train.AdamOptimizer().apply_gradients(gradient)

The error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-22-10282dee2005>", line 10, in <module>
    train_op = tf.train.AdamOptimizer().apply_gradients(gradient)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/optimizer.py", line 605, in apply_gradients
    update_ops.append(processor.update_op(self, grad))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/optimizer.py", line 189, in update_op
    raise NotImplementedError("Trying to update a Tensor ", self._v)
NotImplementedError: ('Trying to update a Tensor ', <tf.Tensor 'strided_slice_9:0' shape=() dtype=float32>)

---

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): v1.10.1-0-g4dcfddc5d1 1.10.1
• Python version: 3.6.5
• Bazel version (if compiling from source): NA
• GCC/Compiler version (if compiling from source): NA
• CUDA/cuDNN version: NA
• GPU model and memory: NA
• Exact command to reproduce: NA

[Hoeze]

I debugged this problem:
Since this variable slice is a Tensor, it gets wrapped by the class _TensorProcessor in tensorflow/python/training/optimizer.py.
Since update_op() is not implemented in this special case, the gradient cannot be applied.

There would be multiple solutions to this problem:

• Make variable slices return another special variable type
• Implement update_op() in _TensorProcessor for tensors supporting some .assign() method

[nellymin]

I cannot quite understand what are the reasons for this to not be implemented?
This, combined with a still unresolved issue #1325 where imported model's variables are not visible makes retraining a pretrained model practically impossible.

[jvishnuvardhan]

@Hoeze Is this still an issue? Thanks!

[Hoeze]

@jvishnuvardhan Yes, the issue is still the same

[jonas-eschle]

Are there any news on this? The implementation of this would be helpful for us as well

[maxhgerlach]

Hi @rmlarsen, is some extension to lift this limitation on the TensorFlow roadmap? Applying an optimizer to a slice of a variable would unblock quite a few more flexible approaches to training certain classes of models.

[mohantym]

@Hoeze !

I was able to replicate the issue in 2.11 and TF Nightly 2.12.0-dev20221215.

Thank you!

[SNMS95]

Are there any updates on this ?

[Emalude]

Hi, any updates about this issue? I'm using a library for Knowledge Graph Embedding models based on Tensorflow to develop an explainability gradient-based method by tracking the influence of single training samples. Given the high number of parameters of such models I think it would be very useful for Tensorflow to support the gradient computation of only a subset of training variables.