2018 CodeSprint

Charles LaPierre edited this page Sep 25, 2018 · 8 revisions


During the 2018 DIAGRAM Code Sprint we will investigate ways to make STEM education more accessible and inclusive for all learners. We also welcome persons with disabilities and learning differences, teachers, advocates, designers, developers, and other experts to help create new educational technologies or enhance existing ones.


Accessibility Lightweight Options Framework

The idea of adjustable site options to support user preferences and personalized accessibility has a long history and several mature manifestations, such as the Fluid Project’s User Interface Options. However, current solutions can be heavyweight in terms of their dependencies and setup, requiring the adoption of particular frameworks or libraries. I’d be interested to experiment with building a very lightweight accessibility options framework without any external dependencies using only modern browser capabilities to manipulate CSS, with the goal of both increasing the ease of adoption of this concept and better behavior in performance-restricted contexts such as low-bandwidth.

Accessible Code Repository

Assembling a collection of best practices examples for common interactions (may of which already exist and need to be collated in a central place) E.g., fluid, what sock, W3C Authoring Practices, synchronized text-to-speech highlighting, carousels (photo wheel), page settings, date picker, Options AT bar for web pages (Contrast, FontSize, GrayScale, etc.), eye gauzing selection and rotation of carousels/wheels etc.


The goal is to make the Nanotation browser extension accessible, and to document the methodology and best practices, so other annotation software (especially Web-based systems) can easily follow suit.

Provides a context menu item that lets you copy a link to a text selection. This link includes a custom query parameter (text) used by this extension. When that custom link is followed, the Nanotation extension decodes the text parameter and searches for that text; if it finds it, it highlights the selection text in the shared document and scrolls to the parent block element for the selection.

Code Block navigation in Atom Nuclide

We will attempt to build an Atom/Nuclide package that provides keyboard navigation shortcuts to traverse code in the editor based on the codemirror-blocks Abstract Syntax Tree. This is a first step in preparing the Atom/Nuclide code editor for integration with assistive technology (like a screen reader) access methods.

Enhanced Image Descriptions

Universally designed method to add either short descriptions and or extended descriptions to images in a way that is both screen reader accessible as well as accessible to everyone with or without assistive technology.

Interactives Simulations

We explored the problem of drag and drop in a web context. We investigated the scope of drag and drop by looking at its different forms throughout certain websites, and across PhET sims. This included a conversation with Taliesin Smith, an accessibility designer/researcher/developer at PhET to discuss the broad extent of drag and drop in a PhET contenxt. We then spent time familiarizing ourselves with Apple's "bag" drag and drop solution, and decided to try to further our understanding of that using PhET sims.

In terms of our code and implementation, we tried to use aria Listbox as a way to achieve this. First we looked at using a group of three listboxes, where one was the bag, and you could move items into the bag, and from the bag into any other group (two in this case). This implementation was not smooth or easy to do with more than a bag and one other group. Mostly because of the complexity of button options based on the number of groups, i.e. "place from bag to group 1" button and "place from bag to group n" button. Currently master of 2018-interactives repo has listbox.html, which shows an implementation working with just one of the "buckets" (not the rope bucket). Here we tried to to have just a single "add" and "take" button combo for each group, but couldn't quite get it to work. The strange names in the page are because we were modeling from the PhET Sim Forces and Motion: Basics.

Next we spent much of the day looking into a more complicated approach that (we thought) could potentially solve the shortcomings of Listbox. We thought that it may be easier to implement a more comprehensive "bag manager," to be thought of as the model for the bag, and then each interaction with a group implemented as a listbox duo (or tuple). For example if there was one bag, and two groups that held items, each group would be set up as two listboxes with operations to grab and place items to/from the bag. So the "bag" (model) is actually two listboxes (view) that show the same content, one for each group. For this example there are four listboxes total. This way you have simpler operations for the interaction, when you focus onto a group, you can add an item to the bag, and there are only two listboxes in that interaction. The bag listbox right next to that particular group also has simple operations that only apply to that group, i.e. "place item(s) from bag into that group." The implementation went south here, because I was too heavily interested in pursuing a Listbox focused approach. Listboxes need focus on them in order to manipulate content, so it made it very difficult for the bag "manager" to keep all of its listbox views in sync. The work done for this approach is in the listbox-tuples branch.

Next steps would be to take this idea and to try to implement something more custom.

Math Decision Tree

Math Detective

  • Ron Ellis (Lead), Andrew Ringler, Mateus Teixeira, Sujeeth Thirumalai, Miguel Juárez, Sean Hastings, with support from Neil Soiffer and Kartik Sawhney
  • Math Detective GitHub Repo

Math Karaoke

We will build a browser-based solution that display math equations and reads them aloud via text-to-speech with synchronized highlighting of the part being read, karaoke-style.

This will be an experimental approach and we'll iterate in search of a successful user experience. Educators and students with experience with dyslexia could provide valuable feedback and guide the approach of the project.

The project doesn't have to involve very advanced math, so no worries if you’re not a math wiz 🙂

This would also be an opportunity to experiment with higher quality text-to-speech reading of math equations, including control of prosody (pause, volume, pitch, intensity) using SSML (if you're into that sort of thing).

At the end of the project, we will have authored some simple content (tutorial on solving quadratic equations?) that will be easier for dyslexic students to access.

Math Speech Input


  • Neil Soiffer (Lead) + domain experts (Ann Gulley, Rebecca Lannan, Bella Simmons, Tom Mcmillen, Beth Powell, Jennifer Larson, Deanna McCusker
  • MathShare GitHub Repo
  • MathShare Demo

Mathshare helps teachers and students overcome everyday challenges ranging from poor handwriting to tracking a student’s work in order to pinpoint where issue areas arise, Digital math platforms often just give students a place to plug in an answer without a way to show their work. Moreover, many of these platforms are not fully accessible. By allowing students to digitally do their work and teachers to see how students are thinking through math problems, Benetech is creating an inclusive math classroom where students of all abilities can work independently, which is critical for their educational and professional success.

Mayan Number Counting

This is a number matching game framework that primarily aims to build a stronger sense of numeracy.

For some students getting a firm grasp of the relationships among numbers, counting, and quantity is challenging. A game that lets students practice matching different representations of number and quantity could be useful. Some teachers like to use aspects of the Mayan number system, in which a bar represents five, and a dot represents a unit, in this connection. Other representations include groups of items, that may be homogenous, or (more challenging) may include items of different kinds.


Extract and identify parts of speech from a phrase with a visual and auditory representation for nouns

Stack Calculator

The idea of this project is to create a flexible workflow that allows a student with limited vision and limited motor coordination to succeed in school.

To make the problem tractable, we focused on one math problem for this sprint.

This idea is to provide programming-like tools to the student, who can use them to create lists of numbers to use as a scratchpad.

This is intended to be a voice interface, but may include physical buttons as needed. Our feeling is that physical buttons will be most useful for:

  • Initiating a voice command
  • Rejecting a voice recognition result
  • Confirming a recognition result.

Distraction Management

  • Mitchell Evan (lead), Andrew Ringler, Travis Snyder, Jingyi Li

Navigation Simplification

  • Kevin Yang (lead), Shane Lian, Enamul Prince Hoque

Text Simplification

  • Rory Soiffer

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.