Keras_NLP Getting Started Tutorial: Mixed Precision Error; AttributeError: 'LossScaleOptimizerV3'

[kmirijan]

Issue Type

Bug

Source

source

Keras Version

Keras 2.15

Custom Code

No

OS Platform and Distribution

5.15.133.1-microsoft-standard-WSL2 on (I think)Ubuntu 18.04.6 LTS

Python version

3.9.18

GPU model and memory

NVIDIA GeForce RTX 3080 Ti Laptop GPU 16GB GDDR6 VRAM

Current Behavior?

I am running the code directly from the Keras_NLP getting started guide.

When I try to instantiate the classifier = keras_nlp.models.BertClassifier.from_preset("bert_tiny_en_uncased_sst2"), I run into an error. AttributeError: 'LossScaleOptimizerV3' object has no attribute 'name'

There is a line in this tutorial that sets a mixed precision policy of mixed_float16. If I comment this out, everything works fine. Why is this error coming up in the documentation?

The only meaningful difference I made to the code is changing os.environ["Keras_Backend"] = 'jax' to os.environ["Keras_Backend"] = 'tensorflow'. I tried it both ways and got the same error and the documentations aid it shouldn't make a difference anyway.

Standalone code to reproduce the issue or tutorial link

This is code directly from https://keras.io/guides/keras_nlp/getting_started/ up to and including the Inference with a pretrained classifier section of the tutorial.

Relevant log output

AttributeError                            Traceback (most recent call last)
Cell In[10], line 1
----> 1 classifier = keras_nlp.models.BertClassifier.from_preset("bert_tiny_en_uncased_sst2")
      2 classifier.predict(["I love modular workflows in keras-nlp"])

File ~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/keras_nlp/src/models/task.py:236, in Task.__init_subclass__.<locals>.from_preset(calling_cls, *args, **kwargs)
    235 def from_preset(calling_cls, *args, **kwargs):
--> 236     return super(cls, calling_cls).from_preset(*args, **kwargs)

File ~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/keras_nlp/src/models/task.py:221, in Task.from_preset(cls, preset, load_weights, **kwargs)
    218     return cls(backbone=backbone, preprocessor=preprocessor, **kwargs)
    220 # Task case.
--> 221 return load_from_preset(
    222     preset,
    223     load_weights=load_weights,
    224     config_overrides=kwargs,
    225 )

File ~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/keras_nlp/src/utils/preset_utils.py:184, in load_from_preset(preset, load_weights, config_file, config_overrides)
    182 if load_weights:
    183     weights_path = get_file(preset, config["weights"])
--> 184     layer.load_weights(weights_path)
    186 return layer

File ~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File ~/miniconda3/envs/tf-gpu/lib/python3.9/site-packages/keras/src/optimizers/optimizer.py:820, in _BaseOptimizer.load_own_variables(self, store)
    817 """Set the state of this optimizer object."""
    818 if len(store.keys()) != len(self.variables):
    819     msg = (
--> 820         f"Skipping variable loading for optimizer '{self.name}', "
    821         f"because it has {len(self.variables)} variables whereas "
    822         f"the saved optimizer has {len(store.keys())} variables. "
    823     )
    824     if len(self.variables) == 0:
    825         msg += (
    826             "This is likely because the optimizer has not been "
    827             "called/built yet."
    828         )

AttributeError: 'LossScaleOptimizerV3' object has no attribute 'name'

[sachinprasadhs]

Hi, Thanks for reporting the issue.

Since we have migrated Keras to support multi backend framework which is Keras 3.

Make sure you have keras 3 and install all the dependency packages in the below mentioned order to run the example .

!pip install tf-keras
!pip install tensorflow-text
!pip install -q --upgrade keras-nlp
!pip install -q --upgrade keras  # Upgrade to Keras 3.

Here is the working Gist for reference

[kmirijan]

Ok, that seemed to work. But I did run into this error during the upgrade of keras to 3.0.4

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.15.0.post1 requires keras<2.16,>=2.15.0, but you have keras 3.0.4 which is incompatible.

I don't really know why this error popped up. It doesn't seem to make sense for keras 3 to be incompatible with the newest stable version of tensorflow that I am using, 2.15.0. Especially since the notebook works on my machine now.

[sachinprasadhs]

if you're installing using requirement.txt file, usually you will end up with this error, since Tensorflow 2.15 installs Keras 2.15 and there will be conflict to this when you install Keras 3.
The solution for this is to install these conflicting packages separately, first install TensorFlow, later install Keras 3.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.

[github-actions]

Are you satisfied with the resolution of your issue?
Yes
No