SimServer, WebStart, and NetLogo in Classrooms

(Generally, consider this page obsolete, as it is a reference to work that I (Jason) did years ago on a system that is really no longer maintained and just kind of keeps running on its own. Should I ever die and something needs to happen with it, you will need all the king's horses and all the king's men to put that broken garbage together again. Godspeed.)

The original draft of this page can be found here.

For ModelSim, many technical problems were faced by researchers, so we, the Devel folks, had to come up with some solutions for them. This page surveys those problems and how they were solved.

Problem #1: How do we distribute NetLogo into schools?

It shouldn't be surprising that the CCL has a desire to see NetLogo models used to facilitate learning in schools. The default way that one might go about this is to install NetLogo, open it, and find the desired model in the NetLogo library. But getting students in a classroom to install NetLogo, locate it on their computer, and open up the correct model is (unfortunately) a non-trivial thing; trying to enforce that workflow in schools leads to classroom management disasters, with lots of class time wasted, frustrated students, frustrated teachers, and frustrated researchers—simply put, no one's happy with that. Instead, a low-friction solution was required to get NetLogo into the hands of the students, immediately ready for them to use.

Since we, on ModelSim, use WISE as our means of delivering classroom content, it would be especially low-friction if we could present NetLogo models directly alongside the WISE content in the browser. There are two prominent in-browser solutions (Java applets and a JavaScript-based NetLogo), but neither is an amazingly pleasant solution at the moment.

First, the JavaScript-based NetLogo is still a ways off from being release-ready, so that's not even an option here, given the imminent need for a means of distributing NetLogo in classrooms. As for applets, throughout 2012 and 2013, Java applets were mired in a panic-inducing tempest of crippling security concerns. To my knowledge, the respective IT staff have had applets disabled on student computers at every school our researchers have visited. There are actually many other reasons why NetLogo applets are less-than-optimal for distribution in schools, but I won't bother listing them all here, as the fact that they tend to be disabled for security reasons is a sufficiently strong nail in the applet coffin to stop that idea dead in its tracks.

With applets and a native browser client out of the question, that severely limits our options and basically means that the content can't be fully delivered in-browser. Somehow, we need to be able to seamlessly distribute the NetLogo Java application in a configurable way, though. Fortunately, there exists a mostly-good solution for that: Java Web Start.

The crux of Web Start's functionality lies in the JNLP file—an XML file that the user downloads to execute through Web Start. Web Start reads the file, gathers from it the necessary information to acquire the desired Java application, and then downloads the .jar files and launches the application. Data can also be specified in the JNLP file, which will then be propagated into the Java application when Web Start launches it. By embedding information into Java application arguments and Java system properties through the JNLP file, NetLogo models can utilize that data to configure how the model is run (i.e. "start up NetLogo with "; "set the code for to the value of the netlogo.dynamic_code property", or virtually any other imaginable NetLogo model configuration that can result from interactions between NetLogo and the outside world). The reading of Java system properties in NetLogo wasn't something that was easily doable in the past, so I created the (admittedly trivial) Props extension to facilitate this.

It's true that there are numerous downsides to Web Start (i.e. long, redundant downloads, and that it takes students away from the rest of their learning content in the browser), but, overall, it has been working pretty well for us.

Problem #2: Configuring JNLPs is a pain! How can we make it easier?

Web Start JNLP files are very configurable, but configurability often brings with it a higher cognitive load on anyone who wants to use that-which-is-highly-configurable—Web Start being no exception. As such, it would be impractical if every activity designer had to know how to correctly write JNLP files for NetLogo. Naturally, we solve this problem by forcing only one person—Jason!—to look at the JNLP documentation and figure out which parameter settings are necessary for deploying NetLogo into a classroom setting, and then creating an easy-to-use JNLP file generator that is highly configurable while still having sensible-for-NetLogo defaults where possible. On top of that, having a JNLP generator also bestows upon us an additional benefit over using static, hand-written JNLPs: we can create JNLPs on-the-fly to send down dynamically-generated data with ease. This makes for a much richer, more collaborative learning experience with NetLogo, when data can flow seamless into, out of, and across Web Start instances.

I won't talk much more about the JNLP generator here, as I already covered it at length in an earlier discussion on netlogo-devel with Reuven Lerner. Please see that post for further details.

Problem #3: I was really inspired by the NSA's work on PRISM! How can I do the same sort of thing with my students?

An excellent question! As you might know, NetLogo has some logging functionality for recording events that occur within the run of a model, but the existing logging functionality is imperfect for deployment of NetLogo into schools; it doesn't give you information specifically tailored to your needs, and that it's difficult to collect the actual log files from students' computers.

To solve the first problem, I created the Custom Logging extension, which has a handful of primitives for logging customized messages, variable values, and global values through NetLogo code. That is, if you want a particular subset of the globals logged every 10 ticks, or for a particular message to be printed out to the logs when the user clicks a button in the model, the Custom Logging extension is the extension for you.

As for the "hard to get logs from students' computers" problem, it has been solved by creating a system that automatically transfers log files from NetLogo to SimServer instances. By including is_logging: true in your JNLP generator request, the JNLP that you receive will be configured to silently run NetLogo's logging functionality while the user runs the model, and then automatically sends the logs off to the SimServer instance when the user goes to close NetLogo. With that, logging is made pretty simple.

Problem #4: How can we lower the setup threshold of HubNet in the classroom?

Coordinating HubNet activities in classrooms can be difficult, and the difficulty is exacerbated by networks that don't permit for HubNet server address broadcasting. Communicating IP numbers to a room full of people can really just be a royal pain. Consequently, I have built into SimServer some functionality that makes coordinating HubNet activities to be a lot easier.

To utilize this functionality, the "teacher" (or whoever is going to run the activity's HubNet server) must make their JNLP generation request to the /jnlp/gen/hubnet/<userName> URL (where <userName> is the username that the teacher/host wants to be known by). This URL is used exactly like the standard JNLP generation URL. As such, sending it the request body

{
  "is_hubnet_server": true,
  "model_url": <modelURL>
}

will get you a JNLP that launches the model located at <modelURL>, just like normal. The only effect of using the special JNLP generation URL's, is that it communicates to SimServer that you plan to soon register your IP address for HubNet.

After that, you download the HubNet server, and the clients can magically connect, right?—WRONG! There's one more step. Since we can't snoop out the host's local IP from an applet (since applets are disabled in the vast majority of schools), and since there's no other sane way to get someone's local IP from within the browser, NetLogo (downloaded through Web Start) has to do the IP-snooping work. Below is the code that ought to be in whatever model you want to use this auto-matchmaking functionality with, which gathers the IP and port and sends them up to the SimServer instance:

extensions [props web crypto snooper]

to register-address

  let teacher-name          props:get "netlogo.registry.teacher_name"
  let url                   props:get "netlogo.registry.register_url"
  let teacher-name-http-key props:get "netlogo.registry.http.teacher_name_key"
  let data-http-key         props:get "netlogo.registry.http.data_key"
  let modulus               props:get "netlogo.registry.public_key.modulus"
  let exponent              props:get "netlogo.registry.public_key.exponent"

  let address (word snooper:local-ip ":" snooper:hubnet-port)
  let encrypted (crypto:rsa-encrypt address modulus exponent)
  let params (list (list data-http-key encrypted) (list teacher-name-http-key teacher-name))

  let response web:make-request url "POST" params
  let response-code first but-first response

  if-else (response-code = "HTTP/1.1 200 OK") [ user-message "HubNet registry succeeded!  You may now tell the clients to connect." ] [ user-message "HubNet registry was a giant failure!  If the problem persists, please reacquire NetLogo from WISE." ]

end

Don't worry about all of the Java properties that are being read; the special JNLP generation URL automatically takes care of getting you all of those. You'll also note that I introduced two new extensions (Crypto and Snooper) for this, but you otherwise need not worry about those, either. Obviously, feel free to customize—or even remove—the text that is displayed on registry success/failure.

Anyway, if you want to use this system, you essentially need to add the above code into the HubNet model in question and make sure that the code gets called. This code cannot be run at startup, because NetLogo's built-in startup block is called without waiting for the person running the server to actually start their HubNet server (and we don't know the HubNet server's port number until the server is fully started). Instead, I recommend having the HubNet model's setup procedure call the above code. On top of that, you can certainly set up some sort of system whereby setup only ever calls register-address the first time that setup is called—in fact, this is recommended.

After this code is called by the server, students/clients may connect. (The way this is done is imperfect and is subject to future improvements.) A lesser way of doing so would be to use the /hubtest URL that SimServer provides, and then have the client fill out the form, listing out his/her desired username, and the username of the host he/she wants to connect to. That's really only intended for internal use, but... maybe you really like that for some reason and insist on using it.... Whatever. In WISE, however, we have a much smoother way of doing this, since the WISE usernames of the students and their teachers are already known, so they can simply click a button/link that directs the browser to /bindstudent?User Name=<clientName>&Teacher%20Name=<hostName>, which will generate the correct JNLP, and, when the HubNet client loads, the user will automatically be logged in as <clientName> and connected to the host with username <hostName>. As such, things are pretty easy for end users, while requiring moderate effort from curricula designers.

Problem #5: Doing activities and looking at logs is all well and good, but what if I actually want my students to create something in NetLogo as classwork? How do I get ahold of their work?

To accommodate for this sort of thing, SimServer has the ability to store and display user work, a simple example of which can be seen here. The work page serves as a place for users to discuss and interact with uploaded work. SimServer also allows supplements to be uploaded along with work (i.e. the NetLogo view images that are associated with the export-worlds on the page linked above), and it allows interactions to be defined for the work and its supplements (i.e. clicking on a view image generates a JNLP that will launch NetLogo and immediately load the model with uploaded export-world state). With all this, it's fairly easy to upload, discuss, collaborate upon, and fork student work. Well, easy for end users, that is; for someone constructing these sorts of activities, a bit more effort is required. For example, to create a work type—that is, something that defines how your work will be displayed and interacted with—you must first go to /work-type/create and enter a name for their desired work type—I know, lame. Then, you will end up at /work-type/edit/<workTypeName>, where you can customize how your work type will be behave. Admittedly both of these pages are fairly primitive, since they aren't intended for use by anyone put power users.

Anyway, the page for editing work types has three inputs: the first allows you to write JavaScript to construct the HTML that will visually represent things of this work type, the second allows you write JavaScript for the do_custom_<workTypeName> JavaScript function, which customizes how users can interact with the work (often as a function that is called by an onclick in the HTML output from the first text area), and the third and final input states the expected file extension for files of this work type.

To see an example, the work type code for export_world is available here. The code in its first input finds a graphical supplement and puts it into an <img> tag, specifying that do_custom_export_world will be called with a particular object as its input when the image is clicked upon. The code in the second input utilizes that object to send a request to the JNLP generator and redirect the browser to download that JNLP. The final input specifies that export_world files ought to be .csv files. Now, clearly this is far from the most simple, easy-to-use aspect of SimServer, but, with its open-endedness, it brings with it great potential for enacting powerful ideas.

That should all explain how to discuss, present, and interact with work, but it still hasn't been made clear how work gets up there, in the first place. For that, we turn to NetLogo and some example NetLogo code that uploads an export-world to a SimServer, and then sends up an export-view as a supplement to it:

extensions [web props]

to upload-to [upload-url]

  let user-description user-input "How would you describe the current state of this model?"
 
  let run-id-tuple    ["run_id"    "apples123"]
  let period-id-tuple ["period_id" "4"]
  let user-id-tuple   ["user_id"   "Alvin"]
 
  let root-url         "http://abmplus.tech.northwestern.edu:9005/" ; Could also pass this down as a system property to avoid hardcoding it into your model
  let root-upload-url  (word root-url "submit-")
  let model-url        (props:get "netlogo.model_source_url") ; Use the Props extension to (hopefully) find out where this model came from
 
  ; Set the upload type as `"export_world"`, and send up the model's URL, so people have a way
  ; of consistently loading this `export-world` against the correct model
  let work-meta-tuple  (list "metadata" (word "{ \"type\": \"export_world\", \"model_url\": \"" model-url "\" }"))
  let work-desc-tuple  (list "description" user-description)
 
  let work-upload-url  (word root-upload-url "work")
  let work-http-method "POST"
  let work-params      (list run-id-tuple period-id-tuple user-id-tuple work-meta-tuple work-desc-tuple)

  ; Export the world to SimServer with all of the necessary parameters
  ; We get back a 2-item list for the `work-response`, which is of form
  ; `[ref_id http_response_status_code]`
  let work-response web:export-world work-upload-url work-http-method work-params
 
  ; Set the `ref_id` from the response; state that the supplement will be of type 'export_view'
  let view-ref-id-tuple (list "ref_id"   (first work-response))
  let view-meta-tuple   ["metadata" "{ \"type\": \"export_view\" }"]
 
  let view-upload-url  (word root-upload-url "supplement")
  let view-http-method "POST"
  let view-params      (list view-ref-id-tuple view-meta-tuple)

  ; Upload the view as a supplement
  let view-response web:export-view view-upload-url view-http-method view-params
 
end

The code is, admittedly, a bit verbose, but the overall gist of it is that it builds two HTTP requests from system properties and the state of the model. The client first streams the export-world output as something that it wants SimServer to interpret as being of the export_world work type. The server sends back an ID in the HTTP response, which the client then uses as a ref ID when sending up the export-view. The Web extension is used here to make HTTP requests and to make it easy to stream a wide variety of data into and out of NetLogo.

For the curious, a more extensive overview of SimServer's "student work" system is available here.