Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Database query expects mutable elements #105

Closed
vojtsek opened this issue Sep 6, 2020 · 1 comment
Closed

[BUG] Database query expects mutable elements #105

vojtsek opened this issue Sep 6, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@vojtsek
Copy link
Contributor

vojtsek commented Sep 6, 2020

Describe the bug
util/multiwoz/dbquery.py:42
The query() method expects elements of the constraints argument to be mutable.
However, tuples are used as the elements and given to the method call in the ConvLab code MultiWozEvaluator._final_goal_analyze().

To Reproduce
The error occurs when Analyzer.comprehensive_analysis() is called and the generated goal contains specific values, see call stack:

File "/lscratch/kulhajon/873944.stallo-adm.uit.no/source/train.py", line 214, in _run_evaluation
    result = analyzer(agent, num_dialogs=self.args.evaluation_dialogs)

  File "/lscratch/kulhajon/873944.stallo-adm.uit.no/source/evaluate.py", line 58, in __call__
    self.comprehensive_analyze(agent, agent.name, total_dialog=num_dialogs)

  File "/home/kulhajon/source/ConvLab-2/convlab2/util/analysis_tool/analyzer.py", line 145, in comprehensive_analyze
    task_success = sess.evaluator.task_success()

  File "/home/kulhajon/source/ConvLab-2/convlab2/evaluator/multiwoz_eval.py", line 298, in task_success
    goal_sess = self.final_goal_analyze()

  File "/home/kulhajon/source/ConvLab-2/convlab2/evaluator/multiwoz_eval.py", line 401, in final_goal_analyze
    match, mismatch = self._final_goal_analyze()

  File "/home/kulhajon/source/ConvLab-2/convlab2/evaluator/multiwoz_eval.py", line 382, in _final_goal_analyze
    query_result = self.database.query(domain, constraints)

  File "/home/kulhajon/source/ConvLab-2/convlab2/util/multiwoz/dbquery.py", line 34, in query
    ele[1] = 'centre'

TypeError: 'tuple' object does not support item assignment

Expected behavior
The method does not throw an error.

Additional context
It's reasonable to use immutable elements for other reasons too.
I've created the Pull Request with suggested fix.

@vojtsek vojtsek added the bug Something isn't working label Sep 6, 2020
@zqwerty
Copy link
Member

zqwerty commented Sep 7, 2020

thanks!

@zqwerty zqwerty closed this as completed Sep 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants