Capture Student Submissions

[nionata]

• Update documentation

Server

• Create submission content type
• Add REDIS docker service
• connect to queue on start
• Override the create controller to add to queue
• Create sandbox endpoints

Compile

• rewrite compile to pull from queue

Automation

• add redis addon to review app create
• build and use compile image

Client

• On upload code
  • create a save
  • POST /submissions with the { day: dayId, workspace: "block code", board: "arduino:avr:uno", sketch: "arduino code" }
• After submitting make sequential GET /submissions/:submission-id till status is COMPLETED (hex will be in the final request response)

[nionata]

Leetcode, a cloud based competitive coding site w/ a multiple language cloud compiler, uses a queuing/worker architecture for compiling and verifying code. From the above screenshot, you can see the the initial submission is a post request and a submission id is returned. From there, /check requests are made to check the status of the submission until it is finally returned. This is the exact method that I am proposing we make our compiler.

Benefits:

• Compile has no database module - we would have to write dataaccess code that already exists in cms to add metrics and save compiles
• Compile has no way of checking if someone has submitted recently - if we had it connected to a db like in cms, it would be trivial
• The compiler has no authentication and can be accessed via HTTP by anyone - cms has all of that build out

By routing all of the submissions through cms, we will be able to easily implement a lot of great features (submission tracking, submission limiting, authentication, etc). The compiler will be inaccessible via http, shrinking our attack surface. Additionally, it will be more scalable to run the compiler as a worker feeding from a shared in-memory queue. To make it even more fault tolerant, we can add a dead-letter queue for jobs that won't compile and we want to revisit later.

I have added this to v1.0 because we will be implementing our authentication in v0.3. By doing this after auth, we can ensure we take advantage of all the benefits of this conversion.

[nionata]

This original schema is not going to work because we don't need the same redundancy with submissions as we do saves. This will be refactored to connect submission with session and activity (day)

[tamargoadam]

The new schema for submissions, referenced by Nick, is as follows:

[nionata]

The bull queue has a progress value (0-100) that we can update and listen to. The server will be listening to global changes to progress and update the submission entry status in the db.

progress -> status 

0 -> CREATED
50 -> COMPILING
100 -> COMPLETED

Submissions are initialized as CREATED

[nionata]

Rough flow for submissions