Error submitting to EvalAI (No examples/example_submission.json exists.)

[veritas9872]

I am currently having a great deal of difficulty uploading my files to EvalAI.

The submission guidelines (See http://fastmri.org/submission_guidelines)
mention "examples/example_submission.json"
for an example of what the submission file to EvalAI should look like.

However, no such file exists on this repo.

Also, the guidelines simply say that the files should be zipped, they do not mention exactly how.

I have currently made a .json file whose contents are identical to the .txt file here.
submission.txt

However, whenever I try to upload my results, it always results in a Status: "Failed" state with the stderr file with the message "Submission file cannot be unzipped".

I have stored my files in Google drive with a student account with shared reading/writing privileges.

I have tried .zip, .tar, .tar.gz files and I have tried having the unzipped file containing a folder and directly opening the .h5 files.

I notice that there are no publicly visible submissions on EvalAI and suspect that several others have had difficulties in uploading/evaluating their files.

I would be most grateful for more detailed instructions and examples of how to submit to the leaderboard.

Many thanks in advance to those who can help out.

[adefazio]

Hi Joonhyung, thanks for reaching out. We have engineers working on fixing the submission process now. I'll have them reach out to you to once we think it's working again so that you can submit.

[veritas9872]

Thank you for your help.

I hope that this is a simple issue.

[anuroopsriram]

Hi Joonhyung: I have looked at your submission file. It looks like the URL in your file ( https://drive.google.com/open?id=1MxE12uXYV54wfJmhlEEvD2QiCifK3eVL) cannot be downloaded directly. When I enter it into the browser, I get a download link which I need to click, but I cannot download it programmatically. For example, try doing: $ wget https://drive.google.com/open?id=1MxE12uXYV54wfJmhlEEvD2QiCifK3eVL and it downloads the Google Drive webpage, which it tries to unzip and fails. We would need a URL to a zip file that can be downloaded programmatically. If you want, for this one time, I can manually download and submit your results within a day. For the future, please make sure the zip is downloadable programmatically. Please let me know if you have any other questions. Anuroop

…

On Wed, Mar 27, 2019 at 8:59 AM Joonhyung Lee ***@***.***> wrote: Thank you for your help. I hope that this is a simple issue. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APlYYJx5_CDlO4rptM0x_qz5koSgY__nks5va5VqgaJpZM4cNWgL> .

[anuroopsriram]

Also, please look at this file (https://s3.amazonaws.com/project-fastmri-private/Baseline+Model+Results/unet_multi.zip) for an example submission zip file.

[anuroopsriram]

Your results are on the leaderboard now: http://fastmri.org/leaderboards. I'll close the issue for now. Please reopen it if you run into any more problems.

[veritas9872]

I will fix that part of my submission.
Thank you for your help.

[PranjalSahu]

When can we expect the example json file to be available ?
I wanted to do a submission but there is not example json available.