Bundle uploading/unpacking taking forever

[Didayolo]

I'm trying to upload the following bundle to Codalab V2 (prod LRI instance):

Mortality challenge bundle.

The uploading seems to work but then the unpacking step never finishes.

[Didayolo]

Update: I tried to upload the bundle on Codalab V1.5 (codalab.org) and it is working well.

[ckcollab]

It looks like the problem was due to this line in the yaml:

# remove me to fix bundle unpacking
competition_docker_image: ''

Was this generated by v1.5 dumps?

Pushing some code in a new branch that makes this error clearer

[Didayolo]

Thank you for your answer.

Bundle are not supposed to be backward compatible?

If no, we should have a script converting from V1.5 yaml to V2 yaml file.

[ckcollab]

If no, we should have a script converting from V1.5 yaml to V2 yaml file.

This happens transparently if you upload a v1.5 bundle.

Was this bundle generated by v1.5 dumps? Trying to understand where the blank argument came from.

[Didayolo]

It worked!

Now I still have another not working bundle. I corrected the competition_docker_image line in the yaml file too but it is not sufficient. There are still no error message when the unpacking takes forever (maybe you didn't push the new change yet).

This one is a competition with 2 tasks (therefore 2 scores). Maybe this is linked to the issue.

Here is the bundle:

Chems challenge bundle.

[Didayolo]

Was this bundle generated by v1.5 dumps? Trying to understand where the blank argument came from

I am not sure. I wrote this yaml file by hand from another example. Maybe the previous example had this argument and I just removed the content inside the quotation mark. Or maybe I took it from a dump and made changes.

[ckcollab]

Got it, since it works on V1.5 we'll make blank competition_docker_image work on v2

[Didayolo]

OK that's perfect.

For the "Chems challenge bundle", the error comes from something else.

[Didayolo]

Still not able to upload the Chems challenge bundle.

[Didayolo]

I've just tried another bundle that works on Codalab V1 but fails on Codalab V2 (also with multiple scores).

Also, I had a bundle failing in both versions but in Codalab V1 I had an error message while in Codalab V2 I just have this infinitely long unpacking.

[jimmykodes]

I will look at implementing a fix for the fact that you aren't getting an error message for this bundle. The error I have found is that the get_starting_kit.html is an empty file, and pages with no content are not considered valid. So if you add some text (or remove the file altogether) the competition should upload correctly. (I have confirmed that removing the file from the yaml will successfully upload on my local v2 instance)

[Didayolo]

Thank you very much for your help. I'll try this as soon as possible.

As for the previous error in this thread (the empty field in yaml file), empty HTML pages didn't cause the bundle to fail in Codalab V1.

For convenience and back compatibility, maybe empty HTML files should be handled.

I will look at implementing a fix for the fact that you aren't getting an error message for this bundle.

Indeed, error message are crucial for debugging bundles.

[jimmykodes]

This should be fixed on the server now.
We have allowed for empty HTML files for compatibility convenience.

[Didayolo]

Indeed it works, thank you very much. I can now upload the Chems bundle.

The last one is Heart Attack challenge bundle.

As always, the unpacking doesn't finish so I don't have an error message. This competition should be very close to "Chems" so I don't see why it isn't working.

[jimmykodes]

This is because your competition.yaml has a space as the first character of your file, which the yaml format reads as an invalid start of file, and bombs out. Removing that space should fix the upload.

While i do not believe it is possible to engineer a solution that will be able to report all of these kinds of edge case bugs to users, I have some ideas I can discuss with Eric to at least prevent the "hanging" of the unpacking step. Even if the best error we can give is, "Something appears to be wrong with your bundle", I think that will be a better solution than just letting it "unpack" indefinitely.

[Didayolo]

Hi Jimmy,

I can't see any space at the beginning of the file:

Moreover, this bundle works in Codalab V1 so the YAML file should be readable.

[jimmykodes]

Apologies. My mistake, not sure how I must have added that space at the begging of the file...

I do not have credentials to log in to the production server to look at logs there so I am limited in my debugging until tomorrow when I can talk w/ eric. However, if I were to guess, can you try removing the blank competition docker image line?

I know this was an issue that was fixed in an earlier commit, but it is possible that was reverted by our most recent patch, fixing blank page content. If this does work, I should be able to have this fixed for real early tomorrow.

[Didayolo]

Awesome, everything works now. Thank you for your efficiency!