You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jan 13, 2022. It is now read-only.
I tried to generate some task data for question 19 with the following default command:
babi_tasks 19 1000 > test_19.txt
In the resulting file I got, I observed some conflicting facts, for example
1 The office is south of the garden.
2 The office is west of the hallway.
3 The kitchen is west of the garden.
4 The garden is south of the office.
5 What is the path from hallway to garden? w,n 1 2
1 and 4 cannot hold at the same time. Another example is
1 The kitchen is west of the garden.
2 The hallway is north of the kitchen.
3 The office is south of the garden.
4 The office is east of the garden.
5 What is the path from kitchen to office? e,s 3 1
Here 3 and 4 are confusing. And one more:
1 The bedroom is west of the bathroom.
2 The garden is north of the bedroom.
3 The hallway is west of the bedroom.
4 The hallway is north of the garden.
5 What is the path from bedroom to garden? w,s 3 4
In this one there are two paths from bedroom to garden, (1) using fact 2; (2) using 3 and 4. The generated ground truth answer ignored the more direct solution, and the two solutions are actually conflicting, as fact 2 says garden is north of the bedroom but fact 3 and 4 together imply that garden is south of the bedroom.
Is this a common problem?
The text was updated successfully, but these errors were encountered:
It seems like there was a bug in the routine for adding decoys, thanks for finding it @yujiali! It was adding location i instead of i + 1 as a decoy, which caused locations to show up twice. I believe the changes in 7a37fb5 should fix it; let me know if you think it's still incorrect.
I tried to generate some task data for question 19 with the following default command:
In the resulting file I got, I observed some conflicting facts, for example
1 and 4 cannot hold at the same time. Another example is
Here 3 and 4 are confusing. And one more:
In this one there are two paths from bedroom to garden, (1) using fact 2; (2) using 3 and 4. The generated ground truth answer ignored the more direct solution, and the two solutions are actually conflicting, as fact 2 says garden is north of the bedroom but fact 3 and 4 together imply that garden is south of the bedroom.
Is this a common problem?
The text was updated successfully, but these errors were encountered: