Skip to content

Student Work Uploads

Jump to bottom Edit New page
Jason Bertsche edited this page Feb 20, 2014 · 24 revisions

Making Uploads

Uploading Work

Uploading user work to a SimSense server is relatively simple. Simply make an HTTP to the /submit-work URL, with the following parameters (optional / required):

  • run_id
    • The identifier for the class's unit's run (specifically in reference to WISE)
  • period_id
    • The identifier for the class period (specifically in reference to WISE)
  • user_id
    • The identifier for the user submitting the work
  • data
    • The data that comprises the "work" that's being uploaded
  • type
    • A label that describes the kind of work being uploaded (optional, but highly recommended; the student should not choose this; the teacher/curriculum designer should force all similar work to be uploaded with the same type, since type can/is used by the SimServer discussion system in determining how to present work, and how to allow people to interact with it); can also be passed in as the type member of the the metadata
  • metadata
    • A JSON object that can contain any extra data that one might like to upload and record. Nothing within this object is mandatory, and it can be essentially hold any string value that the work-/content-designer wants to be able to get ahold of. On a SimServer instance that is being shared for use by multiple different content designers, the metadata is a good place to put uniquely-identifying information about the purpose or content-owner of the uploaded work.
  • description
    • A textual description of the work being uploaded; ideally, something insightful like, "A really cool run of my model, where the sheep seem like they're about to go extinct, but then make a seemingly-miraculous repopulation effort!"

run_id, period_id, and user_id, combined, make up a unique path for looking up submitted work by any given student for any given activity. If not using this system with WISE, feel free to spoof the values, but spoof them consistently.

PLEASE NOTE THAT run_id, period_id, user_id, AND work_type ARE VALIDATED BY THE SERVER. INPUT THAT DOES NOT PASS THE VALIDATION PROCEDURE WILL BE REJECTED BY THE SERVER. ALL OF THESE FIELDS CAN ONLY HAVE VALUES CONTAINING ONLY ALPHANUMERICS, UNDERSCORES, OR HYPHENS, WITH THE EXCEPTION OF work_type, WHOSE VALUES ALSO CANNOT CONTAIN HYPHENS.

Uploading a Supplement

Sometimes, it's desirable to upload some sort of supplement along with your work. A supplement can be virtually anything; the server is flexible is what it will accept, but the work is on you to make the uploaded supplements operable in the discussion system (we'll get to that later in the section on Submission Types).

In order to upload a supplement, you must first upload user work. When you do that, SimServer will reply to your POST with a number. That number is the ref_id of the work that your supplement will need to associate itself with. As such, uploading a work supplement requires POSTing to the /submit-supplement URL with the following parameters:

  • ref_id
    • The ref_id mentioned above, which you get as a reply to a POST request that submits work
  • data
    • The data that comprises the "supplement" that's being uploaded
  • type
    • A label that describes the kind of supplement being uploaded (optional, but highly recommended; the student should not choose this; the teacher/curriculum designer should force all similar supplements to be uploaded with the same type, since type can/is used by the SimServer discussion system in determining how to present supplement, and how to allow people to interact with it); can also be passed in as the type member of the the metadata
  • metadata
    • A JSON object that can contain any extra data that one might like to upload and use server-side

After a supplement is uploaded, attempts to view the work should also allow you to access the any/all supplements that correspond to it.

Example: Submitting Work from NetLogo

Suppose that you wanted to upload work to a SimServer from NetLogo. This might seem like a daunting task, but it's really not too bad with the Web extension. Here's an example that sends up an export-world as the work, and then sends the view up as a supplement:

extensions [web props]

to upload-to

  let user-description user-input "How would you describe the current state of this model?"
  
  let run-id-tuple    ["run_id"    "apples123"]
  let period-id-tuple ["period_id" "4"]
  let user-id-tuple   ["user_id"   "Alvin"]
  
  let root-url         "http://abmplus.tech.northwestern.edu:9001/"
  let root-upload-url  (word root-url "submit-")
  let model-url        (props:get "jnlp.netlogo.model_source_url") ; Use the Props extension to (hopefully, assuming that the JNLP's creator passed down the Java system property `jnlp.netlogo.model_source_url` into the model) find out where this model came from
  
  ; Set the upload type as `"export_world"`, and send up the model's URL, so people have a way
  ; of consistently loading this `export-world` against the correct model
  ; THE SLASHES BELOW SHOULD BE THE OTHER KIND OF SLASH (GitHub doesn't like backslashes for some reason)
  let work-meta-tuple  (list "metadata" (word "{ /"type/": /"export_world/", /"model_url/": /"" model-url "/" }"))
  let work-desc-tuple  (list "description" user-description)
  
  let work-upload-url  (word root-upload-url "work")
  let work-http-method "POST"
  let work-params      (list run-id-tuple period-id-tuple user-id-tuple work-meta-tuple work-desc-tuple)

  ; Export the world to the SimServer with all of the necessary parameters
  ; We get back a 2-item list for the `work-response`, which is of form
  ; `[ref_id, http_response_status_code]`
  let work-response web:export-world work-upload-url work-http-method work-params
  
  ; Set the `ref_id` from the response; state that the supplement will be of type 'export_view'
  let view-ref-id-tuple (list "ref_id"   (first work-response)) 
  let view-meta-tuple   ["metadata" "{ /"type/": /"export_view/" }"]
  
  let view-upload-url  (word root-upload-url "supplement")
  let view-http-method "POST"
  let view-params      (list view-ref-id-tuple view-meta-tuple)

  ; Upload the view as a supplement
  let view-response web:export-view view-upload-url view-http-method view-params
  
end

Work Discussion

At any /students-in/:run_id/:period_id URL, you can view the list of students who have submitted work for the period with :period_id and the run with :run_id. Additionally, at any /work/:run_id/:period_id/:user_id URL, you can view the work submitted by the user with :user_id from the period with :period_id for the run with :run_id. Naturally, you can use the listing from /students_in/:run_id/:period_id to dynamically construct a directory of links to discussion pages of students' work.

(/work/:run_id/:period_id and /work/:run_id are also available for bigger-picture work viewing, and /runs and /periods-in/:run are the less-granular counterparts of /students_in/:run_id/:period_id.)

Once on the discussion page, there are several things worthy of note:

  • The submitted works are listed out here for viewing.
  • There is a non-nested discussion system, whereby you can post comments at the bottom of any work's discussion thread. To do this, simply fill in the fields for the username and comment, and press your keyboard's return key.
  • The page will not be immediately updated in the event of a new comment or work being posted; the page needs to be refreshed from within your browser in order to see updates. Submitting a comment will refresh the page for you.
  • The manner in which work is displayed and can be interacted with can be customized, as detailed in the Submission Types section below.

Batch Work Downloads

One might find it useful to pull down the work submissions as a file tree, rather than trying to analyze all of the work on the discussion pages. As it turns out, SimServer has just the thing for that!

At the /download-work/:run/:period/:id/(:filename).zip URL (and its less-granular, :id-less and :period/:id-less counterpart URLs), work can be obtained in zip file form. The generated zip file will contain a tree structure of runs with periods with users with work submissions with metadata, comments, data, and all supplements. The work is all bundled together in a logical structure for you to analyze at your leisure. It's really just as simple as that.

Submission Types

As described earlier, a submission can—and generally should—have a type label supplied for it. Through assigning a submission a type, we can treat a group of things with the same type label in its own special way on the SimServer, and, when it comes to customizing behavior on the server through use of type labels, there are two important URLs to be aware of: /work-type/create and /work-type/edit/:type.

/work-type/create is the place to initialize a type by supplying a name for it. Simply give a label name for the type and click "Submit". If such a type already exists, your attempt will be rejected. Otherwise, the type will be created and can then be more-specifically defined at the URL /work-type/edit/:type (where :type is the name of the type label that you just initialized). [Note: This workflow is clunky, and is planned to be streamlined one day.]

At the /work-type/edit/:type URL, you are given several input boxes in which to customize the behavior of submissions bearing the type label of :type. The leftmost box is for writing JavaScript that will customize the presentation of your work on page load. The middle box is for writing JavaScript that will customize what happens when someone clicks on the representation of your work. The rightmost box denotes the file extension that uploaded work should be expected to have (e.g. export-worlds from NetLogo are named like MyExport.csv, so, for an export-world type, this box would hold the value csv; GZipped tarball archives are named along the lines of MyTarball.tar.gz, so, for those, the box would contain the value tar.gz).

It would likely help to see an example of how all of this stuff works, so, continuing with our earlier theme, let's say that we want to present NetLogo export-worlds as our work items, and we want to be open-ended and allow the work to be visualizable with an image of the NetLogo view, or of the NetLogo interface. We also want to download the export-world when the view/interface image is clicked.

We can start with the easy part: going to /work-type/create and creating the types export_view, export_interface, and export_world, and then going to their /work-type/edit/:type pages and setting the box for the file extension be csv for export_world, and png for the other two. Furthermore, since export_view and export_interface are just supplements in this case, we don't need to write any JavaScript presentations/actions for handling those types—though, one can imagine a situation in which they are supplements on one page and actually works on another.

Now, we get to the hard part; the task of writing JavaScript for presentation and interaction with an uploaded export-world is quite an open-ended one—and purposefully so, at that!—so let's slow down a bit and take a closer look at the customization functionality.

Graphically Presenting Work

The JavaScript for producing a graphical representation should essentially be a block of code that expects a single non-standard variable to be in scope: the data variable, which has the following (hopefully self-descriptive) members:

  • period_id (String)
  • run_id (String)
  • user_id (String)
  • type (String)
  • data (String)
  • metadata (Object)
  • description (String)
  • supplements (Array of Objects)
  • comments (Array of Objects)

metadata objects are of unspecified form; they contain whatever members the metadata of the work was uploaded with. Objects in the comments array are shaped as follows:

  • timestamp (Number)
  • user_id (String)
  • comment (String)

Objects in the supplements array for submitted works take the following form:

  • type (String)
  • data (String)
  • metadata (Object)

As with submitted works, the metadata for supplements is of unpredictable form.

Now that we know what information is available to us, it's just a simple matter of generating and returning the text for the HTML element that will visually represent the work and make it clickable.

Example: Presenting Work with a Supplementary Image

First, let's find a view or interface image to use for our visualization. Surely, if one exists, it will be in data's supplements array. In searching, though, we have no need to be picky; let's just grab the first matching supplement that we find:

var imageHolderOpt = null;

for (var i=0; i < data.supplements.length; i++) {
  var supplement = data.supplements[i];
  if (supplement.type === "export_interface" || supplement.type === "export_view") imageHolderOpt = supplement;
}

We just iterate through the array until we find a supplement with one of the types that we want.

Now that we have the (hopefully) have the image in imageHolderOpt, let's create an HTML tag for showing the image:

var assetsPath       = "/assets";
var imageNotFoundURL = assetsPath + "/images/not_found.png";

var imageURLOpt = imageHolderOpt ? (assetsPath + "/" + imageHolderOpt.data) : null;
var imageTag    = '<img src="' + (imageURLOpt ? imageURLOpt : imageNotFoundURL) + '" style="width: 100%; height: auto;" />';

assetsPath isn't entirely necessary, but it makes the code a bit cleaner to work with. You'll also notice that we assume the existence of an "Image Not Found" image at the /assets/images/not_found.png URL. After that, by means of the supplement's data member, we extract the image's URL out of it and build the image tag for our supplement (if it exists). We then set the style to take up as much width with the image as we can, and scale the height accordingly.

All that's really left is to set it up so that clicking on the image will trigger our action. But what's going to happen in the action? Presumably, it needs some information about our export-world in order to know how to download the thing, so... how do we get that information to the action? Well, let's build a JavaScript object that contains all of the information that we want the action to be aware of:

var objStr = JSON.stringify({
  path: assetsPath + '/' + data.data,
  model_url: data.metadata.model_url
});

Thinking ahead, we realize that it should need only two things in order to be an effective download of an export-world: the URL at which the export-world can be downloaded, and the URL at which the model can be downloaded. As such, we load those into a JavaScript object, stringify it, and store it into objStr.

Now to make the image clickable and be done with it:

return "<a href='javascript:void(0)' onclick='do_custom_export_world(" + objStr + ")'>" + imageTag + "</a>";

We wrap imageTag inside of the an anchor tag, which links to nowhere, and, instead, runs do_custom_export_world on click, taking our objStr as a parameter. do_custom_export_world is the name of the automatically-generated function that runs the JavaScript for the action for the export_world type. That is, for all work types used on a given page, a do_custom_<type name> function is automatically created, which accepts a single argument: data. In this case, we passed objStr as data. Ideally, we'd want to pass the object directly, rather than as a string, but, since we're building an HTML tag here, it has to be a stringified version of the object; it will automatically be turned back into a JS object when do_custom_export_world is called.

Anyway, here's the fully-assembled version of the code for displaying the export-world with an uploaded view or interface, and running do_custom_export_world when clicked:

var imageHolderOpt = null;

for (var i=0; i < data.supplements.length; i++) {
  var supplement = data.supplements[i];
  if (supplement.type === "export_interface" || supplement.type === "export_view") imageHolderOpt = supplement;
}

var assetsPath       = "/assets";
var imageNotFoundURL = assetsPath + "/images/not_found.png";

var imageURLOpt = imageHolderOpt ? (assetsPath + "/" + imageHolderOpt.data) : null;
var imageTag    = '<img src="' + (imageURLOpt ? imageURLOpt : imageNotFoundURL) + '" class="work_image" />';
var objStr      = JSON.stringify({
  path: assetsPath + '/' + data.data,
  model_url: data.metadata.model_url
});

return "<a href='javascript:void(0)' onclick='do_custom_export_world(" + objStr + ")'>" + imageTag + "</a>";

Allowing Interaction with Uploaded Work

All that remains for us now is to write the code that will be loaded into do_custom_export_world. As mentioned above, this function is autogenerated for us, and, as with the above code, there's a single non-standard variable in scope here: data. data takes whatever form we gave it in the presentation JavaScript when we passed the data forward to do_custom_<type name>. In our case, we included two members: path and model_url. With that, let's walk through the creation of the JavaScript for the action.

Example: Sending Work to NetLogo

First, let's start by making use of the data we loaded into data:

var makeProp = function(name, value) {
  return {
    "name": name,
    "value": value
  };
}

var host           = location.protocol + "//" + location.host;
var export_prop    = makeProp("jnlp.netlogo.world_state_url",  host + data.path);
var model_url_prop = makeProp("jnlp.netlogo.model_source_url", data.model_url);

Since we'll be calling into the JNLP generator to download and run this export-world in NetLogo, it seems most-prudent to send these URLs down to NetLogo through Java system properties. As such, we start by creating a makeProp function for generating a system property in the format expected by the JNLP generator. Then, since the URL in data.path is relative to the root of the SimServer, we create host to clearly store this SimServer's root URL into a variable and prepend it to data.path to generate the full path to the export-world download. After that, it's a simple matter of storing the two full URLs into *_prop variables and moving on.

Now that we have bits and pieces of data assembled, let's combine everything together to create the JSON packet that we're going to send to the JNLP generator:

var upload_data = {
  is_netlogo: true,
  model_url: data.model_url,
  properties: JSON.stringify([export_prop, model_url_prop])
};

It's a vanilla NetLogo application (is_netlogo: true), the model should be loaded from data's model_url (model_url: data.model_url), and we want to send down our two Java system properties into the NetLogo application (JSON.stringify([export_prop, model_url_prop])).

Alright. As it turns out, a version of the jQuery JavaScript library is available from the submission-viewing page, so we can leverage it here to make POSTing to the JNLP generator a lot easier for ourselves. So, now that we have a packet and jQuery in our hands, we just have to send the packet out:

$.post(
  '/jnlp/gen',
  upload_data,
  function(d) {
    window.location = d;
  }
);

We POST upload_data to the JNLP generator, and, when we receive a response from the server, we assume that it's the URL of the generated JNLP, so we redirect the browser there to download it and launch NetLogo through Java WebStart. That's all there is to it.

Here is the fully-assembled version of the code for our on-click action:

var makeProp = function(name, value) {
  return {
    "name": name,
    "value": value
  };
}

var host           = location.protocol + "//" + location.host;
var export_prop    = makeProp("jnlp.netlogo.world_state_url",  host + data.path);
var model_url_prop = makeProp("jnlp.netlogo.model_source_url", data.model_url);

var upload_data = {
  is_netlogo: true,
  model_url: data.model_url,
  properties: JSON.stringify([export_prop, model_url_prop])
};

$.post(
  '/jnlp/gen',
  upload_data,
  function(d) {
    window.location = d;
  }
);

Behind the Scenes

Submission System

All submissions (and submission-related accessories) are routed through the controllers.Submission object. There, the input JSON is parsed into the correct business object and run through a validation sequence. The existing business objects (all in the models.submission package) are:

  • UserWork
  • UserWorkComment
  • UserWorkSupplement
  • TypeBundle

All business objects for subtypes of models.submission.Submission are expected to have companion objects that inherit from models.submission.Parser (or a subtype thereof). There is also a model for metadata (models.submission.Metadata), which can be parsed from a string of JSON. Also, when validating input, it is encouraged that the validation workflow make use of some of the plethora of premade validation methods included in the models.submission.Validator object. The use Parser and Validator greatly streamline the process of parsing and validating the input data into business objects and are for your benefit.

As the parsing/validation process executes, its state is passed around in a Scalaz ValidationNEL, accumulating any/all discovered errors in a NonEmptyList[String]. If things work out well, files are saved to the file system (through SubmissionFileManager), information about them is registered into the database (through SubmissionDBManager), and then an optional "cleanup" method is run in the wake of that. If, instead, there are errors, they are sent back to the request-sender in an HTTP ERROR 400 BAD REQUEST.

The Database Layer

models.submission.SubmissionDBManager manages all interactions between the submission system and the MySQL database that stores the information about received submissions. SubmissionDBManager has several simple methods for retrieving business objects from common sets of parameters, and it is also accompanied by the Submittable and Updatable typeclasses, which contain knowledge of how to make a submission or update to the database for the business objects that are managed therein (UserWork, UserWorkComment (currently no Updatable typeclass), UserWorkSupplement, and TypeBundle). There is a collection of relevant database constants in the accompanying DBConstants object, and AnormExtras contains some handy constructs to simplify common database-related operations.

Work Type-Management System

The work type-management system is relatively primitive and shouldn't be very difficult to figure out. Not much insight can be given into such a thing.

Discussion Page

Any business object that is expected to be used on the discussion page is also be expected to have have a JsonWritable typeclass available for it, which allows the business object to be converted into a Jerkson-based JsObject, and, by extension, into a stringified JavaScript object. Existing instances of such typeclasses are contained within the models.submission.ToJsonCoverters object.

In the viewWork method, the controller (controllers.Submission) handles the construction of JavaScript functions from TypeBundles (retrieved from the database), and supplying placeholder functions where any TypeBundle cannot provide one. The updateAndViewWork method is routed to when a comment is posted from the discussion page, and it handles parsing comments and submitting them into the database, before ultimately returning the page from which the comment was sent.

The view template for the discussion page is located at the path app/views/submissions.scala.html, the CSS at public/stylesheets/submissions.css, and the minor bit of utilized JavaScript is at public/javascripts/submission-events.js (which basically only exists to make comment forms submittable with the return key in Chrome). The view template is broken down into many small functions for dynamically generating and filling parts of UI elements (including laying out placeholder elements to be replaced by the presentation JavaScript output on page load), and it also inscribes into the page the collection of action and presentation JavaScript functions that it receives from the controller.

Add a custom footer
Add a custom sidebar
Clone this wiki locally