Misalignment between LanguageModelFeaturizer and DIETClassifier in Chinese nlu data

[xikaluo]

Rasa Open Source version

3.0.3

Rasa SDK version

3.0.2

Rasa X version

None

Python version

3.8

What operating system are you using?

OSX

What happened?

Hello Rasa team, I met the problem shown in the log during developing a Chinese conversation bot. Looks like there are some misalignment between LanguageModelFeaturizer and DIETClassifier when the language is Chinese.

The pipeline of my config file is

• name: "JiebaTokenizer"
• name: "LanguageModelFeaturizer" model_name: "bert" model_weights: "rasa/LaBSE"
• name: "DIETClassifier" epochs: 5

Command / Request

rasa train --force

Relevant log output

Traceback (most recent call last):
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/engine/graph.py", line 461, in __call__
 output = self._fn(self._component, **run_kwargs)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 904, in train
 self.model.fit(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/utils/tensorflow/temp_keras_modules.py", line 190, in fit
 tmp_logs = train_function(iterator)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 885, in __call__
 result = self._call(*args, **kwds)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 950, in _call
 return self._stateless_fn(*args, **kwds)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3039, in __call__
 return graph_function._call_flat(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1963, in _call_flat
 return self._build_call_outputs(self._inference_function.call(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 591, in call
 outputs = execute.execute(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
 tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [4,12] vs. [4,11]
 [[{{node cond/PartitionedCall/cond_11/else/_298/cond/add_1}}]] [Op:__inference_train_function_52502]

Function call stack:
 train_function

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "/anaconda3/envs/rasa3/bin/rasa", line 8, in 

 sys.exit(main())
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/__main__.py", line 121, in main
 cmdline_arguments.func(cmdline_arguments)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/cli/train.py", line 59, in 

 train_parser.set_defaults(func=lambda args: run_training(args, can_exit=True))
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/cli/train.py", line 91, in run_training
 training_result = train_all(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/api.py", line 105, in train
 return train(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/model_training.py", line 170, in train
 return _train_graph(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/model_training.py", line 237, in _train_graph
 trainer.train(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/engine/training/graph_trainer.py", line 108, in train
 graph_runner.run(inputs=

 {PLACEHOLDER_IMPORTER: importer} 

)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/engine/runner/dask.py", line 106, in run
 dask_result = dask.get(run_graph, run_targets)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 553, in get_sync
 return get_async(
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 496, in get_async
 for key, res_info, failed in queue_get(queue).result():
 File "/anaconda3/envs/rasa3/lib/python3.8/concurrent/futures/_base.py", line 437, in result
 return self.__get_result()
 File "/anaconda3/envs/rasa3/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
 raise self._exception
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 538, in submit
 fut.set_result(fn(*args, **kwargs))
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 234, in batch_execute_tasks
 return [execute_task(*a) for a in it]
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 234, in 

 return [execute_task(*a) for a in it]
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 225, in execute_task
 result = pack_exception(e, dumps)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/local.py", line 220, in execute_task
 result = _execute_task(task, data)
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/dask/core.py", line 119, in _execute_task
 return func(*(_execute_task(a, cache) for a in args))
 File "/anaconda3/envs/rasa3/lib/python3.8/site-packages/rasa/engine/graph.py", line 468, in __call__
 raise GraphComponentException(
 rasa.engine.exceptions.GraphComponentException: Error running graph component for node train_DIETClassifier2.

 

[xikaluo]

Exalate commented:

xikaluo commented:

In addition, if I remove the entity annotation in nlu.yml and 'entities' in domain.yml, this problem no longer appears

[sara-tagger]

Exalate commented:

sara-tagger commented:

Thanks for raising this issue, @degiz will get back to you about it soon

✨

Please also check out the docs and the forum in case your issue was raised there too

🤗

[skeama]

Exalate commented:

liugong commented:

I have the same problem with Rasa 3.0.4, and is there any way to avoid this now?

[xikaluo]

Exalate commented:

xikaluo commented:

I have the same problem with Rasa 3.0.4, and is there any way to avoid this now?

In fact I do not meet this problem again after updating rasa to 3.0.4. I guess maybe explicitly setting "entity_recognition" as "true" could solve the problem?

[skeama]

Exalate commented:

liugong commented:

I have the same problem with Rasa 3.0.4, and is there any way to avoid this now?

In fact I do not meet this problem again after updating rasa to 3.0.4. I guess maybe explicitly setting "entity_recognition" as "true" could solve the problem?

By default "entity_recognition" is set to True, this is my configuration, could you please give me some guidance

• name: "JiebaTokenizer" dictionary_path: "data/jieba/" "intent_tokenization_flag": True "intent_split_symbol": "_" "token_pattern": None
• name: RegexFeaturizer
• name: "LanguageModelFeaturizer" model_name: "bert" model_weights: "bert-base-chinese"
• name: "DIETClassifier" constrain_similarities: True epochs: 100
• name: "EntitySynonymMapper"
• name: "ResponseSelector" epochs: 100 constrain_similarities: True
• name: FallbackClassifier threshold: 0.6

[boydfd]

after one week of understanding and debugging the whole code, I was able to find the root cause:
jieba tokenizer will treat space as a token, which means in jieba, "hey you" will be tokenized as "hey", " ", "you".
But for BERT, it seems to ignore the space, as a result, "hey you" will be tokenized as "hey", "you".

for entity loss inside DIET the shape will mismatch: 4(one more for CLS) vs 3

@xikaluo
You can remove all spaces before the bug is fixed.

[pickwu]

when i remove the block of LanguageModelFeaturizer,the issue disappeared.I guess it is because module jieba and module LanguageModelFeaturizer are incompatible.Although in a certain version, they are compatible,hhhh

[sync-by-unito]

➤ Maxime Verger commented:

💡 Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

➡️ More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.