-
Notifications
You must be signed in to change notification settings - Fork 774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a feature to re-reun a submission by challenge host using UI #2209
Comments
@RishabhJain2018 I'd like to work on this |
@yashdusing You can take this up. |
@RishabhJain2018 I'd like to work on this as well. |
@RishabhJain2018 also, could you please verify the doubts I've asked on gitter please |
Sure! I'll take a look at it soon. |
@RishabhJain2018 So, the challenge host no longer has the access to the annotation file he/she previously submitted. So, the challenge host approaches the admin to fetch the submission annotation file to re-run it on the challenge ? Is that right or am I missing something ? |
I'm not sure why you're talking about annotation file here? |
@RishabhJain2018 the file submitted by the participant which the challenge host wants to re-evaluate (re-run) |
@RishabhJain2018 How about the challenge host having a re-run submission button on the leaderboard itself ? |
Hey, @yashdusing The file submitted by the participant is the
No, since the leaderboard doesn't list all the entries (as participants can make them private), I think it'd be ideal to have it under view all submissions tab.
No, please try to understand the architecture, how it works. You need to pass the submission message to the SQS queue and it'll be processed automatically. I've already implemented an admin feature for it. Please take a look at that. |
@yashdusing I guess publishing the submission parameters to the submission Queue would do the work, a similar kind of approach that we used here. Lines 57 to 70 in 343089d
|
Yes @Ayukha . Thanks for pointing to the code. |
@RishabhJain2018 Thanks for clarifying. I was a little confused and thought I needed to make changes directly to worker.py 😅 |
@RishabhJain2018 could you please check my PRs since most of them are not reviewed. Also, my proposal as well |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This issue has been automatically closed as there is no further activity. Thank you for your contributions. |
Current Behaviour
If a challenge host wants to re-run a submission then the host has to email EvalAI admin to re-run a particular submission from admin.
Expected Behaviour
The challenge host should be able to re-run the submission from the UI.
Please feel free to post the approach first.
The text was updated successfully, but these errors were encountered: