WARNING:absl:Found untraced functions such as lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.

[GF-Huang]

WARNING:absl:Found untraced functions such as lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.

[Saduf2019]

@GF-Huang
We see that the issue template has not been filled, could you please do so as it helps us analyse the issue [tf version, steps followed before you ran into this error or stand alone code to reproduce the issue faced]

Please refer to similar issues and let us know, link, #44938 .

[GF-Huang]

@Saduf2019

version: tf 2.4.1

reproduce:

import tensorflow as tf
from tensorflow import keras as ks

model = ks.Sequential([
    ks.layers.LSTM(units=1, input_shape=(5, 1))
])
model.compile(optimizer='adam', loss='mse', metrics=['mae'])

model.save('models/test')

[Saduf2019]

@GF-Huang
Please let us know if the links help.

[GF-Huang]

Not help.

[Saduf2019]

@GF-Huang
Can you please try with tf-nightly and let us know if you face the issue.

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[ClementViricel]

Hello, I have the same issue in tf.2.4.1 si do not close it

[Saduf2019]

@ClementViricel
Can you please try on nightly and let us know if you still have the issue.

[pedrogasg]

@Saduf2019
This is still happening with tf-nightly and we have a new comment about Keras if we try to use tf.model_save.save

[Saduf2019]

I am able to replicate the issue reported on tf 2.3,tf 2.4 nightly, please find the gist here.

[codekcg23]

TensorFlow Version: 2.4.1

Is this solved ? I have the same issue.

[ClementViricel]

Still happening with tf2.4.1
WARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_4_layer_call_fn while saving (showing 5 of 20). These functions will not be directly callable after loading.

[rivershah]

This issue still persists in tf version: 2.11.0-dev20220830 Going to echo @vaskozl: this is a critical feature and should be working. @Saduf2019 this bug been around a long time. Can you please take a look or change assignee who can please help move this along.

[Mushoz]

Exact same issue for me, with a model using LSTM layers as well. Super frustrating that basic functionality like this does not work.

[aaarrti]

Hi, I still have the same issue with tf version: 2.10.0. Namely, Found untraced functions such as embedding_10_layer_call_fn. I did execute model.predict on a batch of dummy data.

[AntonioDomenech]

I still get the same issue, I'm amazed how long this issue is and it has still not been solved. I also tried saving the model in h5 format, but I get the following error then:

NotImplementedError: Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider saving to the Tensorflow SavedModel format (by setting save_format="tf") or using "save_weights".

[Mas2345]

I'm also getting the issue. Not sure if it has any affect on anything. Can I just ignore it?

I noticed PatternFinder above says that it causes the model to slow to a crawl if loaded. Is that true?

[rerdavies]

Also getting this issue.

I switched from h5 format to tf format because the docs (and the actual observed behavior) suggest that training weights aren't properly saved in h5 format. I get a significant increase in loss at the start of training after reloading from an h5 file.

Using CPU (because it's faster than GPU for this net).

Structure is:

input_1 (InputLayer)           [(1, 1, 12)]         0           []                               
                                                                                                  
 lstm (LSTM)                    (1, 1)               56          ['input_1[0][0]']                
                                                                                                  
 lstm_1 (LSTM)                  (1, 1)               56          ['input_1[0][0]']    
                             ....
                                                                                                  
 lstm_23 (LSTM)                 (1, 1)               56          ['input_1[0][0]']                
                                                                                                  
 concatenate (Concatenate)      (1, 24)              0           ['lstm[0][0]',                   
                                                                  'lstm_1[0][0]',                 
                                                                  'lstm_2[0][0]',                 
                                                 ...

                                                                  'lstm_23[0][0]']                
                                                                                                  
 flatten (Flatten)              (1, 24)              0           ['concatenate[0][0]']            
                                                                                                  
 dense (Dense)                  (1, 1)               25          ['flatten[0][0]']   

The warning is:

   WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses, 
  lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn while saving
   (showing 5 of 48). These functions will not be directly callable after loading.

And then:

WARNING:absl:<keras.layers.recurrent.LSTMCell object at 0x7facadb0c990> has the same name 'LSTMCell' as a built-in Keras object. Consider renaming <class 'keras.layers.recurrent.LSTMCell'> to avoid naming conflicts when loading with `tf.keras.models.load_model`. If renaming is not possible, pass the object in the `custom_objects` parameter of the load function.

repeated 24 times.

The model does actually reload correctly. The performance is not worse (although it was stunningly awful in the first place). Also notable: the model runs with GPU disabled because CPU-only is faster for this model.

However, it does introduce an additional substantial memory leak in addition to the huge memory leak that LTSM units appear to already have. (About 10% of total memory per save).

My model now runs out of memory after 2 epochs instead of 3.

lol. Who the heck is bersbersbers, and why has he downvoted every report of a problem in this thread?

[Psyhich]

Having the same issue, can save in h5 format, however tf still not supported
Tensorflow 2.10.0

[EParisot]

I'm facing the issue too, can I assume my model will still works correctly ? I intend to convert it to TFLite format to run on ESP32 device ...

[AidanDugganCIT]

I'm facing the issue too, can I assume my model will still works correctly ? I intend to convert it to TFLite format to run on ESP32 device ...

I have the same Warning when saving and can confirm the model still works as normal when doing inference

[EParisot]

thank you @AidanDugganCIT
Im very happy @bersbersbers found a meaning to his/her life !

[YinuoWen]

I'm facing the issue too, can I assume my model will still works correctly ? I intend to convert it to TFLite format to run on ESP32 device ...

I have the same Warning when saving and can confirm the model still works as normal when doing inference

'tf' format saved model worked and no warning in tf version==2.7.0 before, but have same warning issue in 2.9.0 and colab. The saved model does can works normally.

[timedcy]

i have the same problem in tf2.10, but change Layer to Model, the warnings will disappear, like

# class Transform(K.layers.Layer):
class Transform(K.Model):

[SuryanarayanaY]

Hi @GF-Huang ,

The warnings that you are getting can be safely ignored.AFAIK It should not be problem if the saved model reloads without error.Please find the attached gist where reloading does not change Model architecture or Parameters.

You can also find such warning in Tensorflow text Tutorial which are safely ignored.You can refer groups.google and SO also which supports Ignoring this warning.

And yes you will not get this warning in .h5 format

If you want to suppress warnings you can use the below code.

from absl import logging
logging.set_verbosity(logging.ERROR)

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[BillRaymond]

I am getting the following warning and it is unclear how to address it. Looking here, I do not see a good resolution so just sharing.

WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _update_step_xla while saving (showing 4 of 4). These functions will not be directly callable after loading.

[yung1231]

I have the same problem
tf version==2.11.0

WARNING:absl:Found untraced functions such as embedding_layer_call_fn, embedding_layer_call_and_return_conditional_losses, multi_head_attention_unit_layer_call_fn, multi_head_attention_unit_layer_call_and_return_conditional_losses, feed_forward_unit_layer_call_fn while saving (showing 5 of 546). These functions will not be directly callable after loading.

[itsmetiwaribk]

I also have the similar problem . If any one have found the solution please share that.

WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, dropout_33_layer_call_fn while saving (showing 5 of 10). These functions will not be directly callable after loading.

INFO:tensorflow:Assets written to: ./saved_model_VIX\assets
INFO:tensorflow:Assets written to: ./saved_model_VIX\assets

[saurabhkumar8112]

this error is still present in tf 2.10, 2.11 and 2.12

[garryyan2]

I had the similar problem when I tried to save the model with model.save("filename"). There were warnings. I couldn't load the model either. When I save the model with model.save("filename.keras"), everything is fine.

[MbProg]

I had the similar problem when I tried to save the model with model.save("filename"). There were warnings. I couldn't load the model either. When I save the model with model.save("filename.keras"), everything is fine.

Same here. I also realized that I have the problem when using tensorflow 2.12, but not with tensorflow 2.10

[garryyan2]

@MbProg Thanks for the information. The interesting thing is that my Tensorflow is 2.10.0, which seems to be the highest version currently supported by (Ana)Conda.

[MCPMH]

Still problem with TF version 2.14.0.

When I learn deep learning model in Ubuntu WSL, 'h5' format is fine for saving and learning.

However, they said it as legacy so I tried 'keras' format.

Then, it made error so I tried 'tf' format.

It is fine for learning but it is super slow for learning.

Every epoch, it makes error as "INFO:tensorflow:Assets written to: DeeplabV3Plus.tf/assets".

Therefore, I have to use 'h5' format again.

As shown above, it occasionally use Cuda. (But that shows not bad using Cuda (It is almost end of epochs), previously Cuda graphs look like multiple spikes and their distance was far) (Please see below)

Taking 1m 50s for 'h5' format and 11m 58s for 'tf' format to learn model.

At the end of learning, Cuda shows continuous using but almost entire process, Cuda doesn't fully use itself.