Google Cloud bucket not working

[mirfan899]

I tried to download the data from give link, but It seems data is not there and I'm getting this error message.

[seanmacavaney]

I’m having the same problem.

[seanmacavaney]

EDIT: Note: now that the files are hosted elsewhere (see #10), these links are no longer active.

@mirfan899, I don't want to host these files perpetually, but I have the original files temporarily mirrored here if you need them:

language	file	md5 hash
de	docs	b0d40bb296c3ec903926243e8397560e
de	queries/train	711b9e9c2163fe07468d6303bbf038f8
de	queries/dev	8d6383b34cd332f5f21c70c3e6a97579
es	docs	c82d3e5998f4cefb5a730a337680fac0
es	queries/train	676e7011d020383422556d2c4b39b67d
es	queries/dev	1359d9516bf5dd9bf732012a25e7f536
fr	docs	941a8e717efb1ab74b9017976c08f73a
fr	queries/train	4c9f45b9c7497d6efb44e593d2f5af4d
fr	queries/dev	a86c46314a0abfb090c542e672bb66e2
id	docs	5e9fd243bfcf160a1177c898796ae53e
id	queries/train	e74bc49a64adf32d105eeace23dc1a58
id	queries/dev	3d5f2261edb985d4fe052ed9e379a42a
it	docs	c87e107fdd99f78a8789c984c96f3e46
it	queries/train	2aa7e48decfbfbaa327aa01f0d16bc3f
it	queries/dev	d9df4b10d892288c81611e7ed74e1549
pt	docs	77260081f0332befa6aa3e6c922b8fb9
pt	queries/train	df2ed4ef0bdb93405ba276a92530fc03
pt	queries/dev	262ce189e3d57059b2795c16db44bb81
ru	docs	27f64ca50b1862d285c53a1bcdc26793
ru	queries/train	e4c09d563124569a632780c3ed1179b2
ru	queries/dev	4d803553d69f967764360570c3e66c84
zh	docs	d5672e88206038d8a3d261987b3bd788
zh	queries/train	21649d058877379bc8184923ab8ec408
zh	queries/dev	83820cac1d3e27a7c911d5116ebce558

[rodrigonogueira4]

Hi @mirfan899 @seanmacavaney, sorry, we are migrating to another storage and the files should be available in a few days. Sorry for the inconvenience.

[mirfan899]

Thanks.

[seanmacavaney]

bump - any updates on this?

[rodrigonogueira4]

Sorry about this taking longer than we expected. This PR contains the new link for the translated datasets: https://huggingface.co/datasets/unicamp-dl/mmarco/tree/main/data/v1.1

In the next couple of days, we will release a new version with better translations, with an improvement of 3-4 points in MRR@10. Stay tuned!

[seanmacavaney]

Thanks! Are you planning to make the v1.0 data available too somewhere for, e.g., those who want to reproduce the results from the original paper or other comparisons folks may want to make between the versions?

[lhbonifacio]

Hi @seanmacavaney . As @rodrigonogueira4 mentioned, the v1.1 data is the one available on this PR.
The upcoming v2.1 data will be released soon.

[seanmacavaney]

Thanks for the clarification about the forthcoming v2.1. It looks like the v1.1 still differs from the original release of the data though in a few cases, which could impact one's ability to reproduce the Chinese and Portugese results from the original paper.

file	v1.0 md5	v1.1 md5
chinese_queries.dev.tsv	83820cac1d3e27a7c911d5116ebce558	9ea0d3e92aaf87d65c07c297893d0ff6
portuguese_queries.dev.tsv	262ce189e3d57059b2795c16db44bb81	4210db124ff1e3b7c803b9cb666c5e44
portuguese_queries.train.tsv	df2ed4ef0bdb93405ba276a92530fc03	c253c476ff1ad1d51bae169cea180acd

(For all the other files, the md5s remained unchanged between v1.0 and v1.1.)

It looks like for Chinese, in some cases there were missing translations, which were changed from empty strings to ".". For Portuguese, the translations look rather different-- was a different translation model used?

Thanks!

[lhbonifacio]

@seanmacavaney Indeed we have some differences in queries.dev and queries.train files.
In some case, we manually corrected some issue on these translated files.
However, as far as we know, these fixes did not cause any impact on reproducing the original results from our paper.
The 6980 queries from queries.dev.small evaluation set were not affected.
We are finishing the PR with all the translated data along with each BM25 run file, so it is possible to achieve the same results.

[seanmacavaney]

Great- thanks!

[lhbonifacio]

Closing this issue as the mMARCO is now available on HF datasets.

[seanmacavaney]

Great, thanks @lhbonifacio! I've added v1, the slightly-revised v1.1, and v2 to ir-datasets as well here: https://ir-datasets.com/master/mmarco.html