SPIKE: Research the work that is needed to support and harden the Canvas integration

[ajpeddakotla]

Purpose

For business development purposes, we need an approximation of the scope of work needed to support and harden the Canvas integration built by @judell. Currently the actual implementation is on the roadmap for the end of quarter 2 of 2017.

Background

We currently have a "prototype" of a Canvas integration running, and we have an significant number of schools, teachers, students that are using this integration so that (at a high level):

1. Teachers can assign annotation homework to their students using Hypothesis and Canvas
2. Students can annotate using Hypothesis and these annotations are displayed in the Canvas app
3. Teachers can review and grade the annotations in Canvas.

There are other features that teachers have asked for and that are possible, but this spike is only concerned with replicating the work that has been done in this prototype.

Current Implementation

To read more about the current implementation of integration, please see the following documentation.

Goal of Spike:

1. For engineering to read through and understand the scope of the current implementation - @judell is the point of contact for any questions.
2. To determine the scope of work that needs to be done by the team in order to support the Canvas integration.
3. I'm time boxing this spike to 1 week of work, as it requires a considerable amount of research work.

@judell it would be helpful if you could post a screencast of a walkthrough of the Canvas app outlining the integration and various features.

[judell]

"a screencast of a walkthrough of the Canvas app" OK, meanwhile here are additional docs (mentioned in the above-linked doc):

• Installing the app: https://docs.google.com/document/d/13FFtk2qRogtU3qxR_oa3kq2ak-S_p7HHVnNM12eZGy8/

• Using the app: https://docs.google.com/document/d/1EvxGoX81H8AWDcskDph8dmu4Ov4gMSkGGXr5_5ggx3I

• The repo: https://github.com/judell/lti/blob/master/lti.py

[nickstenning]

I spent a couple of hours on the train yesterday reading through the various documents Jon has put together about the Canvas integration. Here's a quick summary of my analysis. I should stress that I still want to catch up with Jeremy and Jon individually to check my understanding on a couple of things, but I figured sending this earlier would still be helpful.

Glossary

This is probably all old news to you, but like all of tech, there is a lot of jargon, so for the sake of clarity:

• LMS: Learning Management System - software for students and teachers for course management, assignments, marking, etc.
• LTI: Learning Tools Interoperability - technical standards (developed by a consortium) which define protocols by which services can interoperate with LMSes. LTI is basically a bunch of documents, not any piece of software.
• Canvas: a specific LMS developed by Instructure, which provides various LTI-compatible features.
• LTI Tool: a generic term for any software tool that people want to use with an LTI-compatible LMS, even if it doesn't offer any LTI integration itself.
• LTI Tool Provider: a software interface to an LTI Tool that provides LTI compatibility.

To drastically oversimplify the situation as I understand it, Jon's https://lti.hypothesislabs.com is an "LTI Tool Provider" layer for Hypothesis, specifically targeted at Canvas. The current integration makes use of Canvas custom extensions to the LTI spec and so would need modification for use with other LMSes (Blackboard, etc.).

Scope of work

I've focused on the functionality of the existing LTI tool and what it would take for our team to support this. It seems to me that the work broadly falls into two categories:

1. LTI Tool Provider

A standalone service which provides the interaction layer between an LTI-compatible LMS (AKA a "Tool Consumer") and Hypothesis. I think this would want to remain a standalone service which interacts with both the LMS and Hypothesis APIs.

There are as I understand it currently 4 main interactions provided for by the current tool provider:

1. Assignment creation: where a teacher can select a document (either from within Canvas or out on the web) and create an "assignment" for a class.
2. Assignment participation: where a student can use Hypothesis to annotate the document selected by the teacher.
3. Assignment submission: where a student elects to "submit" their work for marking.
4. Export for marking: the student's submission is processed and presented within a marking tool.

I haven't dug deeply into the specific APIs for each of these interactions yet. It is at least clear that the LTI Tool Provider acts as a sort of OAuth Client of the LMS, but it is more than a bit weird. Canvas itself seems to have a reasonable OAuth v2 API which LMS users can use to "log into" the Tool Provider with their LMS account. But the LTI APIs use OAuth v1, and then only the request signing algorithm, not the authorization flow, so there seems to be no LTI-standard way of communicating user identity from the LMS to the Tool Provider. (See below for more discussion of this.)

From the documentation of the existing integration, it seems that there has been at least some friction on the school end of things getting the tool provider properly installed as an external client, so there have been some workarounds needed to get people onboard. I have questions for Jon and Jeremy about this.

2. Fixes to existing Hypothesis tools

There are probably more of these, but here are a couple of things Jon has had to work around while building https://lti.hypothesislabs.com:

1. Via iframe problems: there is some code in via to prevent "double-injection" when a via-hosted page contains iframes. This currently causes problems iframing a via page itself, which isn't the intended behaviour. Fixing this is unfortunately probably a little more work than it sounds like due to the way Via currently rewrites URLs in the page.

2. Via is slow, so the tool provider is doing a bunch of caching of via pages.

3. The Hypothesis annotation API is currently not correctly usable cross-domain. Specifically, Jon has had to work around a bug in the "update annotation" API CORS support.

Questions

There are many small questions I have about the existing integration, and I'll take these up with Jon and Jeremy separately. I have one "big" question, which is not specifically about reproducing this work, but takes into account something Jeremy said in an email to me the other day:

1. My number one goal with the Canvas app is to provision accounts--plain old H accounts--via the canvas or to in some way leverage the "3rd party accounts" work in relation to Canvas.

This is obviously a HUGE point of friction at the moment, so I totally understand why it's the number one ask.

But I've been reading the LTI specs to understand what the contract is between the LMS (AKA Tool Consumer) and Tool Provider, and it looks like any ability for the LMS to provide student user IDs to the Tool Provider is not part of the LTI specification -- in Canvas it's implemented as a custom extension.

It seems likely that if we're going to try and leverage 3rd-party accounts to make annotating from an LMS friction-free for the students, then the LTI Tool Provider is going to end up acting as a middleman between multiple LMS instances (multiple institutions' Canvas installs) and multiple 3rd-party account configurations in Hypothesis.

Even not taking into account the fact that this is only Canvas (and different LMSes may expose user IDs differently) this is not a trivial proposition.

I'm kind of baffled by the fact that LTI doesn't seem to standardise any way of identifying individual students to LTI Tool Providers, and doesn't seem to require any kind of OAuth API which would give us the ability to offer "Login with my school account" for normal Hypothesis accounts.

Definitely more research needed on this last question.

[judell]

It's been a while since I looked at this so I'll need to refamiliarize in order to have a full discussion, but...

there seems to be no LTI-standard way of communicating user identity from the LMS to the Tool Provider

It's user_id (hyp.is/DeF8tOowEeaDwNdmM7baPw/www.imsglobal.org/specs/ltiv1p0/implementation-guide).

There are as I understand it currently 4 main interactions provided for by the current tool provider:

1 Assignment creation: where a teacher can select a document (either from within Canvas or out on the web) and
create an "assignment" for a class.
2 Assignment participation: where a student can use Hypothesis to annotate the document selected by the teacher.
3 Assignment submission: where a student elects to "submit" their work for marking.
4 Export for marking: the student's submission is processed and presented within a marking tool.

Re: 1, we recently discovered that the same interaction that enables a teacher to choose a PDF or web page in the assignment-creation flow also works in the context of a course reading (https://docs.google.com/document/d/1EvxGoX81H8AWDcskDph8dmu4Ov4gMSkGGXr5_5ggx3I/edit#heading=h.8he8uaugn5cu).

In both cases the user (teacher) interaction looks like this: https://docs.google.com/document/d/1EvxGoX81H8AWDcskDph8dmu4Ov4gMSkGGXr5_5ggx3I/edit#heading=h.vpwn0nd0q58h. The teacher reaches for an External Tool, picks our app, we do the interactive bit about selecting a PDF or web page, then the LMS creates an LTI launch URL in the given context (assignment or reading).

That interactive bit is IMS-defined but not core. However according to Bracken Mosbacker at Lumen, who pioneered LTI -- and was at our OER summit last week -- it'll work in many (but not all) LMSs nowadays.

OAuth ... but it is more than a bit weird

Yes, it's a witches' brew of OAuth 1 and 2, Until we added assignment submission we were only using OAuth2, in two flavors which we present to admin users here: https://docs.google.com/document/d/13FFtk2qRogtU3qxR_oa3kq2ak-S_p7HHVnNM12eZGy8/edit#heading=h.7c5uafec3jep. Either way the LTI app receives the standard user_id launch parameter as mentioned above. I believe that's all we need to auto-provision a mapped H user. Since we're not doing that yet, our use of OAuth 2 is only to acquire an API token, and we only need that to enumerate the PDF files in a course repo.

The API call to enumerate files in the course repo will, of course, need to be abstracted to support multiple LMSs.

When we added assignment submission we had to use OAuth 1 signing for that API as you noted above, which, ¯_(ツ)_/¯ , some kind of legacy thing I guess, nobody I've talked to is willing to admit why.

Also, this is hilarious: https://twitter.com/judell/status/764222224291082241

at least some friction on the school end of things getting the tool provider properly installed as an external client

Best practices dictate path A in https://docs.google.com/document/d/13FFtk2qRogtU3qxR_oa3kq2ak-S_p7HHVnNM12eZGy8/edit#heading=h.7c5uafec3jep, and Instructure recommends it, but it can be problematic in ways we have not yet enough data to fully grok. We've documented what we've learned so far here: https://docs.google.com/document/d/1Yzg6acbq9hqgBR5hV2jBVFozDle6M4bjU6_JFOUKono/edit#heading=h.c5vz54of2l8h.

Teachers hate path A and Instructure, recognizing that, provides path B which is what nearly all of our active instances have used, with no significant problems. However path B is a per-teacher thing, so institution-wide use will require path A.

Even not taking into account the fact that this is only Canvas (and different LMSes may expose user IDs differently)
this is not a trivial proposition.

At least this much seems straightforward to me. The LTI app eventually receives an LTI launch request with an IMS-standard user_id parameter. If we have a namespace for the school (also identified in the LTI launch) we check for an LMS id <-> H id mapping, provision the latter if necessary, and then the LTI app negotiates a token and serves H with auto-login. That should work in any LMS.

fixes

1 Fixing this is unfortunately probably a little more work than it sounds like due to the way Via currently
rewrites URLs in the page

We can live with the status quo here for a while.

2 Via is slow, so the tool provider is doing a bunch of caching of via pages.

The caching is only an incidental outcome of 1: we save locally to rewrite.

3 The Hypothesis annotation API is currently not correctly usable cross-domain. Specifically, Jon has had
to work around a bug in the "update annotation" API CORS support.

That's not part of the LTI app, the workaround serves other purposes, notably the DigiPo toolkit (https://hypothes.is/blog/a-hypothesis-powered-toolkit-for-fact-checkers/) which has now also been adopted by our Euro-journo colleagues. It would be great to fix this so I can stop running and supporting a CORS proxy, though!

The key missing ingredient, as noted here, https://docs.google.com/document/d/1iobDNeP9shOrnKX-oR9Yj9rLeozrHUlaB4BgdM0Ikyg/edit#heading=h.6wjp2hbqihx, is Activity Pages. Assignment submission sends back a URL that represents the student's activity on a document. We use a variant of this shitty tool, http://jonudell.net/h/facet.html?facet=user&mode=documents&search=nickstenning, to accomplish 2 things:

1. Display all annotations included threaded replies

2. Prune the conversations to only threads that involve the student whose participation is being evaluated

It all works, but ideally we would throw away the prototype in favor of an AP view that meets requirements 1 and 2

[ajpeddakotla]

Closing this and breaking the work out in user stories here: https://docs.google.com/document/d/19dM-f4pScrhqmpHk6QHg36P3pVu6QTZ6wULmc_1ZTfM/edit#heading=h.uyoufthy3adg