Reproducing results shown in paper

[antonyscerri]

Hi

Could you described the steps to reproduce the results in your paper for OIE 2016 and the other systems. You used the supervised-oie repo and the benchmark included in there. There is also a related repo from the same author oie-benchmark with the same evaluation. However they are slightly different and have had updates and neither seems to work out of the box when running the eval.sh script which should produce the PR Curve plots (part of what i was interested in comparing).

Looking at them its unclear which version of the benchmark corpus is being used and whether you picked test (or dev) split or used the whole dataset. The default script seems to use all the data but when going through the tasks you provide it seems to use the test split.

Thanks

Tony

[MikeDoes]

Hello Gentlemen,
I am also looking at replicating this evaluation procedure for novel models in my master thesis. This code uses .txt files and the supervised_oie uses .conll files. Perhaps, would it be possible to share these txt files with us?
Best Regards.
Michael

[jesseLiu2000]

Hi MikeDoes!
The .txt file will generate automatically in evaluation python, and when you run the stage 3, it will call this python file.

[jesseLiu2000]

Hi antonyscerri !
You can just follow README file to run the scripts one by one with the setting you choose, it will automatically call other python files and script files, you don't need to run them by yourself. Here provide a link to the oie corpus, so it will follow the updates. More, here shows that they combine the files: (https://github.com/cgraywang/deepex/blob/a4a4cf60c96e1bfe3ddc8007498bf5ed783af730/scripts/manager.py#L17)