Added Submission Queue to handling bulk of submission #23
Conversation
| "message": score} | ||
|
|
||
| else: | ||
| # Solve bruteforcing... |
There was a problem hiding this comment.
Hnn, let's not merge this for now. I have some questions. (a) this looks like the queue is processed at the call to the API. What happens if immediately after a submission, the connection is broken. Do we have a nohup type scenario so that the queue remains processing?
If things are left in the queue, we will probably need a cron/daemon to process the queue, no?
Also, how is the student going to get "real time" feedback if we are going to be doing async queue processing?
And we will have to do some rigorous resting before this is merged. Nice work on this but I have had some headaches with such queues and I don't want to have to handle them on prod =]
There was a problem hiding this comment.
Every time student submits the assignment, their submission will be processed on the spot. However if number of current processing is equals to available cores then submission will set to status of QUEUE and student will be showed this message "Your submission is added to queue, you can view result in web interface." (Student will not be able to submit any submission until his/her submission is in QUEUE or EXECUTING state).
Another column is added to Course (Submission History) section which indicates the status of their submission.
After the successful processing of submission result is returned, plus a Thread is started for handling QUEUE submissions. If there is some item in QUEUE then there will a EXECUTING process running also which will handle the QUEUE item later with Thread.
Yes, there should be done some rigorous testing before moving this to prod.
There was a problem hiding this comment.
Well, if there's something in the queue and connection is broken, does it keep processing? Doesn't happen in the normal cases unless you took care of it somehow.
There was a problem hiding this comment.
In case Django server is stopped then this case need to be handle :) will look into this.
There was a problem hiding this comment.
Don't think it even needs the whole server to stop. As far as my understanding goes, if the client is disconnected, all threads for that are stopped. (Anyway, let's worry about other stuff first.)
|
|
||
| # Start thread for testing queued submissions | ||
| # NOTE: There could be max thread "multiprocessing.cpu_count()" threads eg 8. | ||
| Thread(target=run_test_for_next_queue).start() |
There was a problem hiding this comment.
@recluze This is the Thread which is called after submission is processed.
|
It is 108 commits behind, merging this could be destructive. Closing this request. |
No description provided.