Fix text NER evaluation

[siddalmia]

Following the PR in #5. I started testing the evaluation pipeline for the text NER and it seems to also be quite flaky. I have fixed most of the errors but there are some critical ones that I mention in #6.

If you could make the edits to this PR for handling the remaining todos mentioned in the issue. That would be great.

I have currently kept my previous PRs also committed to this one but happy to remove them once you merge those.

Thanks
Sid

[ankitapasad]

LGTM

[ankitapasad]

LGTM

The conflicts are mostly because some of the suggested changes are already accounted for in the updated repo but on different lines. Let me know if there's any case otherwise.

@sshon-asapp @fwu-asapp

[sshon-asapp]

@siddalmia Given that PR #10 merged, can you update the conflict files?

[fwu-asapp]

Hi @siddalmia @sshon-asapp
Is there any reason for changing "datasets" to "dataset" in this PR? If there is no special reason, I would just keep it as "datasets"; Otherwise, it may affect other people who have cloned the repo and are using it.

[siddalmia]

Hi @siddalmia @sshon-asapp Is there any reason for changing "datasets" to "dataset" in this PR? If there is no special reason, I would just keep it as "datasets"; Otherwise, it may affect other people who have cloned the repo and are using it.

Yeah it's actually because huggingface which is needed for your baselines has a library called datasets. I have detailed it here - #5

[fwu-asapp]

@siddalmia Thanks for the clarification, nice catch! Do you mind to change it to data instead? Just to avoid people accidentally typing "dataset" into "datasets" or vice versa.
@sshon-asapp what do you think?

[sshon-asapp]

Good idea. Let move away from dataset and datasets, and use data.

[fwu-asapp]

I renamed "datasets" to "data" in this PR #14. @siddalmia would you please resolve the conflicts? We can merge your PR once there are no conflicts left.

[siddalmia]

@fwu-asapp I will do it soon.

But this is really bad practice. The PRs that were merged before this one were all working out of this PR. So ideally this PR should have been merged before in order to avoid merge conflicts.

[sshon-asapp]

@siddalmia I feel sorry about this. As you may know, this toolkit is in a very early state right now, and the original authors are fixing it as quickly as possible to provide a better experience. Thank you for you understanding.

[siddalmia]

@sshon-asapp @fwu-asapp I have merged this PR to the latest commit.

[fwu-asapp]

@siddalmia Thanks!
@ankitapasad and @sshon-asapp If there is not additional concerns, we can merge this branch.

[ankitapasad]

LGTM