Exception ignored in: function CapturableResource.__del__ after upgrading to TF 2.5

[shkarupa-alex]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS X 11.2.3
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): v2.5.0-rc3-213-ga4dfb8d1a71 2.5.0
• Python version: 3.8.6
• Bazel version (if compiling from source): No
• GCC/Compiler version (if compiling from source): No
• CUDA/cuDNN version: No
• GPU model and memory: No

Describe the current behavior
After upgrading from TF 2.4.1 to TF 2.5.0 i see strange messages in console when run training/testing of my model. See logs below.

Describe the expected behavior
No ignored exceptions should be printed

Contributing - Do you
want to contribute a PR? (yes/no): No

Standalone code to reproduce the issue
https://colab.research.google.com/drive/1IpmIj3WI17j6JkfgGWH0YEpowja7m71Y?usp=sharing

Other info / logs
In some cases i got only such logs

Exception ignored in: <function CapturableResource.__del__ at 0x15ad1c790>
Traceback (most recent call last):
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/training/tracking/tracking.py", line 277, in __del__
    self._destroy_resource()
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 924, in _call
    results = self._stateful_fn(*args, **kwds)
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3022, in __call__
    filtered_flat_args) = self._maybe_define_function(args, kwargs)
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3279, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
AttributeError: 'NoneType' object has no attribute '__wrapped__'

But in another, it prepended with another one:

  Traceback (most recent call last):
    File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 2560, in get_attr
      pywrap_tf_session.TF_OperationGetAttrValueProto(self._c_op, name, buf)
  tensorflow.python.framework.errors_impl.InvalidArgumentError: Operation 'raggedconcat_assert_equal_3_assert_assertguard_placeholder_1' has no attr named '_class'.
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/training/tracking/tracking.py", line 277, in __del__
      self._destroy_resource()
  ...
    File "/Users/alex/.pyenv/versions/3.8.6/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 672, in wrapped_fn
      out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  AttributeError: 'NoneType' object has no attribute '__wrapped__'

[bhack]

Can you share a very very minimal standalone code example that we could copy, paste and run or a Colab to reproduce this?

[shkarupa-alex]

Here is a sample code to reproduce issue
test.py.zip

Also i made a demo in colab https://colab.research.google.com/drive/1IpmIj3WI17j6JkfgGWH0YEpowja7m71Y?usp=sharing
Note, that it should be runned from console, not from notebook

As far as i noticed, the issue gone if code from "main()" function is moved to the top level (drop "def main" line and remove tabs in body code)

[UsharaniPagadala]

@shkarupa-alex
I reproduced the code shared with no errors in tf-nightly.please find the gist.Thanks

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[UsharaniPagadala]

@shkarupa-alex

Please let us know if the issue still persists.Thanks

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No

[jesford]

I am seeing this exception get printed as a warning quite frequently, and found this example of it also showing up in the Tensorflow docs:
https://www.tensorflow.org/tfx/tutorials/transform/census?hl=el#put_it_all_together_2

Exception ignored in: <function CapturableResource.__del__ at 0x7ff6dd76f050>
Traceback (most recent call last):
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/training/tracking/tracking.py", line 277, in __del__
    self._destroy_resource()
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 924, in _call
    results = self._stateful_fn(*args, **kwds)
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3022, in __call__
    filtered_flat_args) = self._maybe_define_function(args, kwargs)
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3289, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
AttributeError: 'NoneType' object has no attribute '__wrapped__'

Is there a way to actually suppress this, if it should in fact be ignored?