AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

[fox88-tw]

提问时请尽可能提供如下信息：

基本信息

• 你使用的操作系统: colab
• 你使用的Python版本: 3.6
• 你使用的Tensorflow版本: 2.3.0
• 你使用的Keras版本: 2.3.1
• 你使用的bert4keras版本: 0.8.8
• 你使用纯keras还是tf.keras:
• 你加载的预训练模型:bert

核心代码

# 请在此处贴上你的核心代码。
# 请尽量只保留关键部分，不要无脑贴全部代码。

from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
import numpy as np

config_path = '/content/drive/My Drive/roberta_zh_L-6-H-768_A-12.zip (Unzipped Files)/bert_config.json'
checkpoint_path = '/content/drive/My Drive/roberta_zh_L-6-H-768_A-12.zip (Unzipped Files)/bert_model.ckpt.data-00000-of-00001'
dict_path = '/content/drive/My Drive/roberta_zh_L-6-H-768_A-12.zip (Unzipped Files)/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True) # 建立分詞器
model = build_transformer_model(config_path, checkpoint_path) # 建立模型，加載權重

输出信息

# 请在此处贴上你的调试输出

---

AttributeError Traceback (most recent call last)
in ()
8
9 tokenizer = Tokenizer(dict_path, do_lower_case=True) # 建立分詞器
---> 10 model = build_transformer_model(config_path, checkpoint_path) # 建立模型，加載權重
11

9 frames
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in is_tensor(x)

AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

自我尝试

不管什么问题，请先尝试自行解决，“万般努力”之下仍然无法解决再来提问。此处请贴上你的努力过程。
網路上，解決問題的方法有二：

1. 由 “form keras.xxx import xxx” 變更為 “form tensorflow.keras.xxx import xxx” ..... (X)
2. keras / tensorflow版本 衝突 ，安裝 相互 不衝突 的版本....(X) 即使 版大推薦的 keras2.3.1/ tf 1.14版本也無法

[bojone]

你的意思是keras2.3.1 + tf 1.14还是报同样的错误？

[cmd23333]

我把tensorflow升级到2.3.1
bert4keras升级到0.9.1
Keras 2.4.3(提示警告要小于2.3.1)
不过这么做就不报错了。。

[bojone]

• 支持tf+keras和tf+tf.keras，后者需要提前传入环境变量TF_KERAS=1。

• 当使用tf+keras时，建议2.2.4 <= keras <= 2.3.1，以及 1.14 <= tf <= 2.2，不能使用tf 2.3+。

• keras 2.4+可以用，但事实上keras 2.4.x基本上已经完全等价于tf.keras了，因此如果你要用keras 2.4+，倒不如直接用tf.keras。

[zhangkaihua88]

我把tensorflow升级到2.3.1
bert4keras升级到0.9.1
Keras 2.4.3(提示警告要小于2.3.1)
不过这么做就不报错了。。

感谢，真是神奇，终于可以在kaggle上用gpu训练了23333

[holoodst]

把TensorFlow降级到2.0.0可以解决这个问题

[xiaoguzai]

tensorflow == 2.4.0rc0
keras == 2.3.1
代码如下：

from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
import numpy as np
import os
os.environ["TF_KERAS"] = '1'
config_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_config.json'
checkpoint_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_model.ckpt'
dict_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True)  # 建立分词器
model = build_transformer_model(config_path, checkpoint_path)  # 建立模型，加载权重
token_ids, segment_ids = tokenizer.encode(u'语言模型')
print('\n ===== predicting =====\n')
print(model.predict([np.array([token_ids]), np.array([segment_ids])]))

另外我将layers以及models中的keras改变为tensorflow.keras
仍然报错

AttributeError Traceback (most recent call last)
in
9
10 tokenizer = Tokenizer(dict_path, do_lower_case=True) # 建立分词器
---> 11 model = build_transformer_model(config_path, checkpoint_path) # 建立模型，加载权重
12
13 # 编码测试

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build_transformer_model(config_path, checkpoint_path, model, application, return_keras_model, **kwargs)
2323
2324 transformer = MODEL(**configs)
-> 2325 transformer.build(**configs)
2326
2327 if checkpoint_path is not None:

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build(self, attention_caches, layer_norm_cond, layer_norm_cond_hidden_size, layer_norm_cond_hidden_act, additional_input_layers, **kwargs)
89 ]
90 # Call
---> 91 outputs = self.call(inputs)
92 self.set_outputs(outputs)
93 # Model

~/enter/lib/python3.8/site-packages/bert4keras/models.py in call(self, inputs)
99 """
100 # Embedding
--> 101 outputs = self.apply_embeddings(inputs)
102 # Main
103 for i in range(self.num_hidden_layers):

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply_embeddings(self, inputs)
453 z = self.layer_norm_conds[0]
454
--> 455 x = self.apply(
456 inputs=x,
457 layer=Embedding,

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply(self, inputs, layer, arguments, **kwargs)
160 self.attention_scores = a
161 return o
--> 162 return self.layers[name](inputs, **arguments)
163
164 def get_inputs(self):

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in symbolic_fn_wrapper(*args, **kwargs)
73 if _SYMBOLIC_SCOPE.value:
74 with get_graph().as_default():
---> 75 return func(*args, **kwargs)
76 else:
77 return func(*args, **kwargs)

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in call(self, inputs, **kwargs)
444 # Raise exceptions in case the input is not compatible
445 # with the input_spec specified in the layer constructor.
--> 446 self.assert_input_compatibility(inputs)
447
448 # Collect input shapes to build layer.

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in assert_input_compatibility(self, inputs)
308 for x in inputs:
309 try:
--> 310 K.is_keras_tensor(x)
311 except ValueError:
312 raise ValueError('Layer ' + self.name + ' was called with '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_keras_tensor(x)
693 ```
694 """
--> 695 if not is_tensor(x):
696 raise ValueError('Unexpectedly found an instance of type ' + 697 str(type(x)) + '. '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_tensor(x)
701
702 def is_tensor(x):
--> 703 return isinstance(x, tf_ops._TensorLike) or tf_ops.is_dense_tensor_like(x)
704
705

AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

[nicken]

tensorflow == 2.4.0rc0
keras == 2.3.1
代码如下：

from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
import numpy as np
import os
os.environ["TF_KERAS"] = '1'
config_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_config.json'
checkpoint_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_model.ckpt'
dict_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True)  # 建立分词器
model = build_transformer_model(config_path, checkpoint_path)  # 建立模型，加载权重
token_ids, segment_ids = tokenizer.encode(u'语言模型')
print('\n ===== predicting =====\n')
print(model.predict([np.array([token_ids]), np.array([segment_ids])]))

另外我将layers以及models中的keras改变为tensorflow.keras

仍然报错
AttributeError Traceback (most recent call last)
in
9
10 tokenizer = Tokenizer(dict_path, do_lower_case=True) # 建立分词器
---> 11 model = build_transformer_model(config_path, checkpoint_path) # 建立模型，加载权重
12
13 # 编码测试

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build_transformer_model(config_path, checkpoint_path, model, application, return_keras_model, **kwargs)
2323
2324 transformer = MODEL(**configs)
-> 2325 transformer.build(**configs)
2326
2327 if checkpoint_path is not None:

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build(self, attention_caches, layer_norm_cond, layer_norm_cond_hidden_size, layer_norm_cond_hidden_act, additional_input_layers, **kwargs)
89 ]
90 # Call
---> 91 outputs = self.call(inputs)
92 self.set_outputs(outputs)
93 # Model

~/enter/lib/python3.8/site-packages/bert4keras/models.py in call(self, inputs)
99 """
100 # Embedding
--> 101 outputs = self.apply_embeddings(inputs)
102 # Main
103 for i in range(self.num_hidden_layers):

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply_embeddings(self, inputs)
453 z = self.layer_norm_conds[0]
454
--> 455 x = self.apply(
456 inputs=x,
457 layer=Embedding,

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply(self, inputs, layer, arguments, **kwargs)
160 self.attention_scores = a
161 return o
--> 162 return self.layers[name](inputs, **arguments)
163
164 def get_inputs(self):

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in symbolic_fn_wrapper(*args, **kwargs)
73 if _SYMBOLIC_SCOPE.value:
74 with get_graph().as_default():
---> 75 return func(*args, **kwargs)
76 else:
77 return func(*args, **kwargs)

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in call(self, inputs, **kwargs)
444 # Raise exceptions in case the input is not compatible
445 # with the input_spec specified in the layer constructor.
--> 446 self.assert_input_compatibility(inputs)
447
448 # Collect input shapes to build layer.

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in assert_input_compatibility(self, inputs)
308 for x in inputs:
309 try:
--> 310 K.is_keras_tensor(x)
311 except ValueError:
312 raise ValueError('Layer ' + self.name + ' was called with '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_keras_tensor(x)
693 ```
694 """
--> 695 if not is_tensor(x):
696 raise ValueError('Unexpectedly found an instance of type ' + 697 str(type(x)) + '. '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_tensor(x)
701
702 def is_tensor(x):
--> 703 return isinstance(x, tf_ops._TensorLike) or tf_ops.is_dense_tensor_like(x)
704
705

AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

应该是keras 版本太低了 更新到2.4.3版本上试试

[InsaneLife]

tensorflow == 2.4.0rc0
keras == 2.3.1
代码如下：

from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
import numpy as np
import os
os.environ["TF_KERAS"] = '1'
config_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_config.json'
checkpoint_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_model.ckpt'
dict_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True)  # 建立分词器
model = build_transformer_model(config_path, checkpoint_path)  # 建立模型，加载权重
token_ids, segment_ids = tokenizer.encode(u'语言模型')
print('\n ===== predicting =====\n')
print(model.predict([np.array([token_ids]), np.array([segment_ids])]))

另外我将layers以及models中的keras改变为tensorflow.keras

仍然报错
AttributeError Traceback (most recent call last)
in
9
10 tokenizer = Tokenizer(dict_path, do_lower_case=True) # 建立分词器
---> 11 model = build_transformer_model(config_path, checkpoint_path) # 建立模型，加载权重
12
13 # 编码测试

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build_transformer_model(config_path, checkpoint_path, model, application, return_keras_model, **kwargs)
2323
2324 transformer = MODEL(**configs)
-> 2325 transformer.build(**configs)
2326
2327 if checkpoint_path is not None:

~/enter/lib/python3.8/site-packages/bert4keras/models.py in build(self, attention_caches, layer_norm_cond, layer_norm_cond_hidden_size, layer_norm_cond_hidden_act, additional_input_layers, **kwargs)
89 ]
90 # Call
---> 91 outputs = self.call(inputs)
92 self.set_outputs(outputs)
93 # Model

~/enter/lib/python3.8/site-packages/bert4keras/models.py in call(self, inputs)
99 """
100 # Embedding
--> 101 outputs = self.apply_embeddings(inputs)
102 # Main
103 for i in range(self.num_hidden_layers):

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply_embeddings(self, inputs)
453 z = self.layer_norm_conds[0]
454
--> 455 x = self.apply(
456 inputs=x,
457 layer=Embedding,

~/enter/lib/python3.8/site-packages/bert4keras/models.py in apply(self, inputs, layer, arguments, **kwargs)
160 self.attention_scores = a
161 return o
--> 162 return self.layers[name](inputs, **arguments)
163
164 def get_inputs(self):

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in symbolic_fn_wrapper(*args, **kwargs)
73 if _SYMBOLIC_SCOPE.value:
74 with get_graph().as_default():
---> 75 return func(*args, **kwargs)
76 else:
77 return func(*args, **kwargs)

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in call(self, inputs, **kwargs)
444 # Raise exceptions in case the input is not compatible
445 # with the input_spec specified in the layer constructor.
--> 446 self.assert_input_compatibility(inputs)
447
448 # Collect input shapes to build layer.

~/enter/lib/python3.8/site-packages/keras/engine/base_layer.py in assert_input_compatibility(self, inputs)
308 for x in inputs:
309 try:
--> 310 K.is_keras_tensor(x)
311 except ValueError:
312 raise ValueError('Layer ' + self.name + ' was called with '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_keras_tensor(x)
693 ```
694 """
--> 695 if not is_tensor(x):
696 raise ValueError('Unexpectedly found an instance of type ' + 697 str(type(x)) + '. '

~/enter/lib/python3.8/site-packages/keras/backend/tensorflow_backend.py in is_tensor(x)
701
702 def is_tensor(x):
--> 703 return isinstance(x, tf_ops._TensorLike) or tf_ops.is_dense_tensor_like(x)
704
705

AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

查看了源码，应该在导入之前就设置TF_KERAS=1。所以代码改成：

import os
os.environ["TF_KERAS"] = '1'
from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
import numpy as np

config_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_config.json'
checkpoint_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/bert_model.ckpt'
dict_path = '/home/xiaoguzai/下载/chinese_L-12_H-768_A-12/vocab.txt'

tokenizer = Tokenizer(dict_path, do_lower_case=True)  # 建立分词器
model = build_transformer_model(config_path, checkpoint_path)  # 建立模型，加载权重
token_ids, segment_ids = tokenizer.encode(u'语言模型')
print('\n ===== predicting =====\n')
print(model.predict([np.array([token_ids]), np.array([segment_ids])]))

[KingfaLuis]

很棒，我有了一些思路，我不过还在整理当中

[jh-lau]

手动将TF_KERAS设成‘1’就行了。

[OveSteve]

环境：3.6.13，TF2.3.0，使用tf.keras
碰到这个错误是在jupyter notebook中，预先忘记在最开始加载设置os.environ["TF_KERAS"]='1'，导致了上面的错误。
补上后再运行发现还是出同样的错误，于是查到了这里。转念一想，可能是之前加载的设置（没有tf.keras=1）还存在内存中，重新运行cell并没有更新，于是整个jupyter重启，清空内容，从tf.keras=1开始运行，成功！