Make batchwise adding of evaluation data possible

[bogdankostic]

When trying to add evaluation data containing millions of documents, my computer ran out of memory.

To solve this problem, this PR adds a method to add eval docs batch-wise so that not all eval docs have to be loaded to memory at once. Furthermore, this PR introduces the method squad_json_to_jsonl as the required format for add_eval_data_batchwise is jsonl.

[bogdankostic]

All tests pass on my local machine. Not sure why test_faiss_index_save_and_load fails in the CI. The error message is sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) disk I/O error, so I suspect that the problem might be in the CI.

[tholor]

I like it. Would propose a different design though.

[tholor]

LGTM!

[Rob192]

Hello,
This PR is not compatible with Windows and throws the following AttributeError: 'WindowsPath' object has no attribute 'endswith'
Would it be possible to e.g. make use of the pathlib Library to prevent this ?
Thanks !

[Timoeller]

Hey @Rob192 sure, we already use Pathlib in some parts of haystack.

Would you like to create a PR with the proposed changes? I guess the change will be minor. Otherwise we can also take over.

[Rob192]

OK @Timoeller ! will do !

[lalitpagaria]

@Timoeller Raised ticket #714 to catch these platform dependent issue early. Currently Windows users are 15%-20% of overall haystack downloads.