Colab: When Imports and Definitions Failed

[andywolf-0921]

colab url
2024/4/29 11:29 +08:00 is working
2024/5/2 16:50 +08:00 is not working
The output is the below.

step 1, full output: MT3 - Setup Environment - output.txt
step 2, full output: MT3 - Imports and Definitions - output.txt

step 1, screenshot:

step 2, screenshot:

TypeError: register_with_handler() got an unexpected keyword argument 'for_restore'
The command import tensorflow.compat.v2 and t5x both failed.
Anyone face the issue, and how to fix it, thanks.

[jmuru]

same issue

[0xdevalias]

This is the error I got:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-2-30d8629039fb>](https://localhost:8080/#) in <cell line: 16>()
     14 import seqio
     15 import t5
---> 16 import t5x
     17 
     18 from mt3 import metrics_utils

3 frames
[/usr/local/lib/python3.10/dist-packages/t5x/checkpoints.py](https://localhost:8080/#) in <module>
   2076 
   2077 
-> 2078 @ocp.args.register_with_handler(
   2079     DatasetCheckpointHandler, for_save=True, for_restore=True
   2080 )

TypeError: register_with_handler() got an unexpected keyword argument 'for_restore'

• https://github.com/google-research/t5x

  • T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales.

This is where that line of code appears to be in t5x:

• https://github.com/google-research/t5x/blob/e84aab9a14be816beae651314217e39fc419fea8/t5x/checkpoints.py#L2078-L2080

@ocp.args.register_with_handler(
    DatasetCheckpointHandler, for_save=True, for_restore=True
)

ocp is imported from orbax.checkpoint:

• https://github.com/google-research/t5x/blob/e84aab9a14be816beae651314217e39fc419fea8/t5x/checkpoints.py#L53

import orbax.checkpoint as ocp

ChatGPT explanation:

The @ocp.args.register_with_handler is a decorator provided by the orbax.checkpoint module. This particular decorator is used to register a function or a method with a specific handler, in this case, DatasetCheckpointHandler, with parameters indicating it should be used both for saving (for_save=True) and restoring (for_restore=True). The function or method that would be decorated with this would then be associated with those operations related to saving and restoring datasets within the context defined by Orbax, typically a framework for managing operations in machine learning workflows.

• https://github.com/google/orbax

  • Orbax provides common utility libraries for JAX users.

  • https://github.com/google/orbax/blob/32522e88cbcc98e7ac9ec9ad0e812aa0436615a2/checkpoint/orbax/checkpoint/checkpoint_args.py#L74-L98

From the colab dependencies install logs:

Collecting t5x@ git+https://github.com/google-research/t5x#egg=t5x (from mt3==0.0.1)
  Cloning https://github.com/google-research/t5x to /tmp/pip-install-rjz7xbnz/t5x_d0982f5011f24beea4d9d45972477f10
  Running command git clone --filter=blob:none --quiet https://github.com/google-research/t5x /tmp/pip-install-rjz7xbnz/t5x_d0982f5011f24beea4d9d45972477f10
  Resolved https://github.com/google-research/t5x to commit e84aab9a14be816beae651314217e39fc419fea8
  Preparing metadata (setup.py) ... done

..snip..

Requirement already satisfied: orbax-checkpoint in /usr/local/lib/python3.10/dist-packages (from flax@ git+https://github.com/google/flax#egg=flax->mt3==0.0.1) (0.4.4)

• https://github.com/google/orbax/blob/main/checkpoint/CHANGELOG.md#044---2023-11-29

---

See also:

• Error trying to import t5x google-research/t5x#1550
  • Use newer Orbax CheckpointManager API in T5X. This change is expected to be a no-op. Also removes a test case that is already covered in Orbax core. google-research/t5x#1547

[Apple-jun]

same issue. could anyone help? would greatly appreciate any help.

[0xdevalias]

It seems like upstream was broken, as per this issue:

• Error trying to import t5x google-research/t5x#1550

Within that issue, they mention that the commit before the one they mentioned seemed to work; so presumably this could be fixed by forcing that package to the version before the error was introduced (and then potentially any related dependencies/etc)

[garzon]

!pip install --upgrade orbax-checkpoint==0.5.8 works as a workaround for me.

[0xdevalias]

Not sure if this is relevant/related, but when I was applying this fix/workaround:

!pip install --upgrade orbax-checkpoint==0.5.8 works as a workaround for me.

Originally posted by @garzon in #151 (comment)

I got the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.15.0 requires ml-dtypes~=0.2.0, but you have ml-dtypes 0.4.0 which is incompatible.
tensorflow-cpu 2.15.0.post1 requires ml-dtypes~=0.2.0, but you have ml-dtypes 0.4.0 which is incompatible.
Successfully installed ml-dtyp

Documenting here in case it is relevant.

---

After that fix, now getting a new issue unfortunately:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-6-689ff6dbf16f>](https://localhost:8080/#) in <cell line: 25>()
     23 note_seq.play_sequence(est_ns, synth=note_seq.fluidsynth,
     24                        sample_rate=SAMPLE_RATE, sf2_path=SF2_PATH)
---> 25 note_seq.plot_sequence(est_ns)

2 frames
[/usr/local/lib/python3.10/dist-packages/bokeh/core/has_props.py](https://localhost:8080/#) in _raise_attribute_error_with_matches(self, name, properties)
    373             matches, text = sorted(properties), "possible"
    374 
--> 375         raise AttributeError(f"unexpected attribute {name!r} to {self.__class__.__name__}, {text} attributes are {nice_join(matches)}")
    376 
    377     def __str__(self) -> str:

AttributeError: unexpected attribute 'plot_width' to figure, similar attributes are outer_width, width or min_width

Which seems to be at least these 2 issues:

• error when running the "Transcribe Audio" stage #144
• Error when transcribing audio in google colab #150

Edit: And you can see my exploration, the upstream issue, and a workaround; here:

• error when running the "Transcribe Audio" stage #144 (comment)

[iansimon]

Sorry about the lack of activity! Colab should be fixed now...