-
Notifications
You must be signed in to change notification settings - Fork 9
Interactive Training Example
To use this example you must have the following Apprentice Learner repositories installed:
- AL_Core - (https://github.com/apprenticelearner/AL_Core)
- AL_Train - (https://github.com/apprenticelearner/AL_Train)
Another approach to training an agent is to manually provide it with problems to solve and then to interactively tutor it in solving these problems. We have applied this approach to training as a means of exploring authoring models to drive intelligent tutoring systems. Specifically, an educational technology designer can train an agent similar to how they would train another person—by providing problems, examples, and feedback. In response, the agent learns rules that can then be used within an intelligent tutoring system to provide one-on-one feedback to human students for arbitrary practice problems.
To see this interactive training approach in action, navigate to the directory where you cloned AL_Train and run the following command:
altrain free_author.jsonYou should see a new browser tab open with a blank multi-column addition problem. (Note the interactive training example currently does not work in Firefox, we recommend using Chrome instead) Go ahead and fill in an initial state for a problem like this (pick any two 3-digit numbers you like):
Then press the “Start State Done” button. Now you can begin demonstrating the steps for doing this problem to the agent. For each demonstration put the correct answer in the box and then press Enter. Then you will be prompted to select the interface elements from which that answer was computed. These are called foci of attention. For example in the first step of this problem we would put a “4” in the right-most box on the bottom, and select the two boxes above that one. When you are done press Next and demonstrate the next step and so on. The first three steps are shown below.
Once you have completed demonstrating a full problem click the Done button to start a fresh problem. Then set the start state for another problem. This time AL will attempt to solve the problem with the knowledge it learned from the previous problem, and will ask for your feedback and guidance as needed. You should see that the agent will propose the first step of this new problem which it has never seen before.
On the bottom left-hand side of the screen you will see all of the actions that the agent thinks it can take displayed in the skill panel. You can click on one of these actions to stage it as the next action (and see it on the tutor display). When an action is staged you can press “YES” to give the agent positive feedback or “NO” to give the agent negative feedback. You may also toggle the ✔ and ✖ symbols and press the “Submit” button to give feedback for multiple actions at once. However, note that you can only apply an action by pressing the “YES” button.
You may also want to sometimes demonstrate alternative solution orders/strategies. For example, for this assignment we’ll want to allow students to do the add (the number placed under the line) or carry (the 1 put above the next column) in any order. We should generally try to demonstrate how to do problems in all reasonable correct ways, and try to give the agent feedback in a broad range of situations.
At each step of a problem it is a good idea to flip through all of the actions that the agent believes can be taken to make sure that 1) All of the correct actions are shown 2) No incorrect actions are shown. If there turns out to be a correct action missing from the set of actions proposed by the agent you can demonstrate a correct action at any point (remember to select the foci before pressing next).
Note, it is possible to select foci incorrectly and cause one of the agent’s learning mechanisms to become confused. If this happens you should see an enormous number of proposed actions show up in the skill panel. Currently, there is no undo button, so you will either need to spend several problems providing counter-training or restart training with a fresh agent. If you refresh the page at any point a new agent will be created and the old one will be lost forever. If you give positive or negative feedback when you should have given the opposite kind of feedback, you can usually override this feedback by doing the same problem over again, so there is no need to restart unless things get really bad.
Sometimes chrome will cache the JSON files that are used during interactive training (e.g., free_author2.json). To overcome this you can rename the json file, or load chrome's development console and disable caching.
Our interactive training software currently has issues with Firefox, we recommend just using Google Chrome instead. By default the training script will open the your default browser. If you have both installed and don’t want to change your default browser you can tell altrain which browser to use by providing a -b argument like this:
altrain free_author.json -b "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"