Skip to content

Selection Algorithms

Octav Popescu edited this page Nov 12, 2026 · 11 revisions

The TutorShop Learning Management System, which was built to deploy CTAT tutors on the web, groups individual tutor instances, or "problems" (a problem is a pairing of a student interface and a model, either a behavior graph or a cognitive model), into problem sets at runtime. Each problem set has a selection algorithm that determines which of its problems will be shown to the student and in what sequence.

TutorShop has a number of built-in problem selection algorithms an author could choose from. Besides that it also allows authors to implement their own custom selection algorithms and link them to the system in one of several ways, as described below and further detailed in Interface to Custom Problem Selectors.

One factor that plays into the built-in problem selection algorithms is the number of times each problem has already been run, in case problem repeating is allowed. All built-in algorithms have a preference for selecting problems that have been run fewer times.

The built-in algorithms can be classified into 3 main groups: simple ordering algorithms, mastery algorithms, and custom algorithms.

Simple Ordering Selection Algorithms

These algorithms are based on the order of the problems in the problem set. They only choose among problems in the problem set that have not been attempted yet, or among problems that have been attempted the fewest times.

The algorithms are:

  • Sequential Order: choose the first problem from the subset mentioned above.
  • Random Order: choose a problem randomly from the subset mentioned above.

Mastery Selection Algorithms

These algorithms aim to select the problem that could advance the student furthest toward mastery. Mastery is hard-coded to 0.95 probability. The algorithms compute a score for each problem, which is a weighted average of the missing skill probabilities. Where a missing skill probability is the difference between the mastery level (0.95) and the current skill p-known, and the weights are given by the number of opportunities for each skill. Problems exercising skills that have already been mastered are not considered.

The computed score is further adjusted by dividing it by the number of times the problem has already been attempted plus 1 (to avoid division by zero). This gives a preference to problems that have been attempted fewer times.

The algorithms are:

  • Sequential Mastery Learning - Harder First: choose the first problem from the subset of problems with the highest score,
  • Sequential Mastery Learning - Easier First: choose the first problem from the subset of problems with the lowest score.
  • Random Mastery Learning - Harder First: choose a problem randomly from the subset of problems with the highest score.
  • Random Mastery Learning - Easier First: choose a problem randomly from the subset of problems with the lowest score.

Custom Selection Algorithms

TutorShop also provides a way for authors to customize the problem selection, by replacing the built-in ones with their own. This could be implemented either as a script running in the browser or on the same server, or an internet service running on any other server.

Tutorshop defines an API for communicating with the custom problem selection service. TutorShop sends a POST request with the student model and some data about the problems to choose from in JSON format, and the service returns a JSON object containing the next problem choice using whatever strategy it wants, together with an updated student model.

The 3 types of custom problem selection algorithms are:

  • Custom Selection Algorithm - Browser: the custom selection algorithm is implemented as a script running in the browser, and is loaded by the TutorShop client when the problem set is loaded. The script is executed in the browser when a problem needs to be selected, and it returns the next problem choice and an updated student model to the TutorShop client.
  • Custom Selection Algorithm - Server: the custom selection algorithm is implemented as a script running on the same server as TutorShop, and is loaded by the TutorShop server when the problem set is loaded. The script is executed on the server when a problem needs to be selected, and it returns the next problem choice and an updated student model to the TutorShop server, which then sends it to the TutorShop client.
  • Custom Selection Algorithm - Internet Service: the custom selection algorithm is implemented as an internet service running on any server, and is called by the TutorShop server when a problem needs to be selected. The service receives a POST request with the student model and some data about the problems to choose from in JSON format, and it returns a JSON object containing the next problem choice and an updated student model to the TutorShop server, which then sends it to the TutorShop client.

For details about the TutorShop API to custom problem selectors see Interface to Custom Problem Selectors.

Clone this wiki locally