Conflicting facts in generated task data

[yujiali]

I tried to generate some task data for question 19 with the following default command:

babi_tasks 19 1000 > test_19.txt

In the resulting file I got, I observed some conflicting facts, for example

1 The office is south of the garden.
2 The office is west of the hallway.
3 The kitchen is west of the garden.
4 The garden is south of the office.
5 What is the path from hallway to garden?      w,n     1 2

1 and 4 cannot hold at the same time. Another example is

1 The kitchen is west of the garden.
2 The hallway is north of the kitchen.
3 The office is south of the garden.
4 The office is east of the garden.
5 What is the path from kitchen to office?      e,s     3 1

Here 3 and 4 are confusing. And one more:

1 The bedroom is west of the bathroom.
2 The garden is north of the bedroom.
3 The hallway is west of the bedroom.
4 The hallway is north of the garden.
5 What is the path from bedroom to garden?      w,s     3 4

In this one there are two paths from bedroom to garden, (1) using fact 2; (2) using 3 and 4. The generated ground truth answer ignored the more direct solution, and the two solutions are actually conflicting, as fact 2 says garden is north of the bedroom but fact 3 and 4 together imply that garden is south of the bedroom.

Is this a common problem?

[bartvm]

It seems like there was a bug in the routine for adding decoys, thanks for finding it @yujiali! It was adding location i instead of i + 1 as a decoy, which caused locations to show up twice. I believe the changes in 7a37fb5 should fix it; let me know if you think it's still incorrect.

[yujiali]

Looks like the problem is fixed, thanks!