You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Concept-map checks: shape tools in the Drawing Pad and a graded map question
#726
This follows up on #479, mainly the sketch format and its question 4 (whether vision grading of drawings is good enough).
Some topics are about how ideas connect, not about one fact. For those, a good check is "draw the map": put the concepts in boxes, connect them with arrows and say what each arrow means. A learner who can only recognize the right option among four usually can't build the map from scratch.
Today this doesn't work well:
The Drawing Pad only has the freehand pen, so boxes and arrows are hard to draw with a mouse or trackpad.
The drawing goes to the teacher as an image, so grading depends on how well the model reads it.
Proposal
Two parts that don't depend on each other.
1. Shape tools in the Drawing Pad. Add Line, Rectangle, Ellipse, Arrow and Text next to Pen, Eraser and Hand. The first four can be generated from a drag (start to end) as ordinary pen strokes. The Stroke type in strokes.ts stays the same, so Eraser, Undo, Clear and the image export keep working. Text is the only tool that needs a new kind of history entry. This part helps any freehand sketch, not only concept maps.
2. A concept-map question type. A plugin that adds a new check where the answer is a map, saved as data (boxes and labeled links) instead of only an image. The teacher gets a per-item grade, so the verdict doesn't depend on vision.
The learning journey
The example below comes after the minibatch/unbiasedness explanation in a gradient descent session. The learner believes a bigger batch reduces bias, which is the misconception the check should catch.
1. The teacher asks for a map
The teacher decides a check needs the learner to produce the structure, not pick it. It poses a concept-map check with the question text and a model map. The model map is part of the question but stays hidden until the learner submits, the same way Coding Questions hides modelSolution.
The learner sees the check card with an empty canvas. In this screen the learner can:
read the question;
open the canvas;
start writing in Your thinking.
Submit stays disabled until the map has at least one box.
2. The learner builds the map
The canvas opens over the session body, with the question at the bottom for reference. The top bar stays visible.
In this screen the learner can:
draw boxes or ellipses for the concepts, and write the label inside with Text;
connect two concepts with an arrow and label the arrow with Text;
use the Pen for anything that isn't a shape, like underlining or circling;
move around with Hand or the right mouse button, and use Undo, Eraser or Clear;
press Done, X or Escape to go back to the question. The map stays as a draft.
To save the map as data, an arrow whose ends touch two shapes becomes a link between them, text inside a shape becomes its label, and text next to an arrow becomes the link's label. Anything that doesn't fit these rules stays in the drawing but isn't graded.
3. The learner reviews and submits
Back in the question, the card shows a preview of the map with an Edit map button. The learner writes their reasoning in Your thinking, the same field as in the other checks, and submits.
The saved answer has the boxes, the labeled links and Your thinking. A PNG of the canvas can go with it, so the teacher also sees freehand marks and the layout.
4. Grading and the teacher's reply
On submit, the plugin compares the map with the model map and marks each item:
a box or link that matches the model map is right (green);
a box or link that contradicts it is wrong (red);
a box or link from the model map that the learner left out is shown dashed as missing.
The header shows a summary like "4 of 5 concepts · 3 of 4 links". The learner can open the model map. The teacher receives the answer, Your thinking and the grading, and its next reply explains what to fix. Here it connects the wrong box ("Bias of ĝ > 0"), the wrong link ("reduces") and the missing box ("Unbiased: E[ĝ] = g"), since they are the same misconception.
How it fits the plugin API
As far as I can tell, part 2 doesn't need host changes. Coding Questions already follows the same pattern:
registerQuestionType with a teach variant, so the teacher can pose it as a check. The questionSchema carries the model map; the answerSchema carries boxes and links.
registerQuestionRenderer: mountAnswerEditor for the preview and Edit map button, mountAnswered for the graded map and the model map toggle.
ctx.ui.openSurface for the canvas over the session body.
gradeAnswer for the comparison. The result goes to mountAnswered and to the teacher.
headerContainer with ctx.ui.mountBadge for the summary, like the test result badge in Coding Questions.
registerInstructions to tell the teacher when a map check fits and how to write the model map.
Plugins import only the type-only API, so this plugin can't reuse the Drawing Pad canvas at runtime. It would need its own canvas code, which is why the two parts are independent.
Embed Excalidraw as the canvas. It has more tools, but it's a large React dependency with its own data model, and its scene format would still need the same box/link extraction.
Open questions
Matching the learner's labels to the model map needs semantic judgment ("Variance of ĝ" vs "Noise in ĝ"). The backend has ctx.inference.complete and gradeAnswer is async, but the question contract says grading has "no jobs, queue or extra LLM pass". Is a plugin allowed to call a model inside gradeAnswer, and with which purpose? If not, gradeAnswer can only match exact labels, and the per-item verdict would have to come from the teacher's reply.
Plugin questions don't have the confidence slider that native checks have. Is that intentional, or could a plugin question opt in?
Should part 1 be a PR to the Drawing Pad, or start as a separate plugin?
These are mockups, not a prototype. I made them and mapped the plugin API with help from Claude Code. If the direction makes sense, I'd start with part 1, since it's small and useful on its own.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Use case
This follows up on #479, mainly the sketch format and its question 4 (whether vision grading of drawings is good enough).
Some topics are about how ideas connect, not about one fact. For those, a good check is "draw the map": put the concepts in boxes, connect them with arrows and say what each arrow means. A learner who can only recognize the right option among four usually can't build the map from scratch.
Today this doesn't work well:
Proposal
Two parts that don't depend on each other.
1. Shape tools in the Drawing Pad. Add Line, Rectangle, Ellipse, Arrow and Text next to Pen, Eraser and Hand. The first four can be generated from a drag (start to end) as ordinary pen strokes. The
Stroketype instrokes.tsstays the same, so Eraser, Undo, Clear and the image export keep working. Text is the only tool that needs a new kind of history entry. This part helps any freehand sketch, not only concept maps.2. A concept-map question type. A plugin that adds a new check where the answer is a map, saved as data (boxes and labeled links) instead of only an image. The teacher gets a per-item grade, so the verdict doesn't depend on vision.
The learning journey
The example below comes after the minibatch/unbiasedness explanation in a gradient descent session. The learner believes a bigger batch reduces bias, which is the misconception the check should catch.
1. The teacher asks for a map
The teacher decides a check needs the learner to produce the structure, not pick it. It poses a concept-map check with the question text and a model map. The model map is part of the question but stays hidden until the learner submits, the same way Coding Questions hides
modelSolution.The learner sees the check card with an empty canvas. In this screen the learner can:
Submit stays disabled until the map has at least one box.
2. The learner builds the map
The canvas opens over the session body, with the question at the bottom for reference. The top bar stays visible.
In this screen the learner can:
To save the map as data, an arrow whose ends touch two shapes becomes a link between them, text inside a shape becomes its label, and text next to an arrow becomes the link's label. Anything that doesn't fit these rules stays in the drawing but isn't graded.
3. The learner reviews and submits
Back in the question, the card shows a preview of the map with an Edit map button. The learner writes their reasoning in Your thinking, the same field as in the other checks, and submits.
The saved answer has the boxes, the labeled links and Your thinking. A PNG of the canvas can go with it, so the teacher also sees freehand marks and the layout.
4. Grading and the teacher's reply
On submit, the plugin compares the map with the model map and marks each item:
The header shows a summary like "4 of 5 concepts · 3 of 4 links". The learner can open the model map. The teacher receives the answer, Your thinking and the grading, and its next reply explains what to fix. Here it connects the wrong box ("Bias of ĝ > 0"), the wrong link ("reduces") and the missing box ("Unbiased: E[ĝ] = g"), since they are the same misconception.
How it fits the plugin API
As far as I can tell, part 2 doesn't need host changes. Coding Questions already follows the same pattern:
registerQuestionTypewith a teach variant, so the teacher can pose it as a check. ThequestionSchemacarries the model map; theanswerSchemacarries boxes and links.registerQuestionRenderer:mountAnswerEditorfor the preview and Edit map button,mountAnsweredfor the graded map and the model map toggle.ctx.ui.openSurfacefor the canvas over the session body.gradeAnswerfor the comparison. The result goes tomountAnsweredand to the teacher.headerContainerwithctx.ui.mountBadgefor the summary, like the test result badge in Coding Questions.registerInstructionsto tell the teacher when a map check fits and how to write the model map.Plugins import only the type-only API, so this plugin can't reuse the Drawing Pad canvas at runtime. It would need its own canvas code, which is why the two parts are independent.
Alternatives
Open questions
ctx.inference.completeandgradeAnsweris async, but the question contract says grading has "no jobs, queue or extra LLM pass". Is a plugin allowed to call a model insidegradeAnswer, and with whichpurpose? If not,gradeAnswercan only match exact labels, and the per-item verdict would have to come from the teacher's reply.These are mockups, not a prototype. I made them and mapped the plugin API with help from Claude Code. If the direction makes sense, I'd start with part 1, since it's small and useful on its own.
All reactions