AI Tutor - set up how first 3 prototypes interact #55015

Erin007 · 2023-11-17T00:33:37Z

The goal is to start 2024 with 3 working prototypes for AI Tutor: 1.) Code Compilation 2.) Validation and 3.) General Chat.

This early work sets up a possible mechanism for how those prototypes can co-exist in the same space. UI is NOT final! Currently no one externally can see any of the AI Tutor panel content. We just need a place to iterate and inch toward resolving some of the ambiguity of what AI Tutor might eventually evolve into.

In the AI Tutor panel, I added checkboxes so a student can indicate what kind of help they'd like to get from the AI Tutor. It's a single select - only one option is possible at a time.

On non-coding CSA levels, the only option is the generic "I have a question" which when selected just shows "Coming soon!"

On non-validated coding levels, there are options for "Code compilation" and "I have a question":

If "Code compilation" is selected, but the code has not yet been run, the student is prompted to run their code:

If "Code compilation" is selected, the code has been run, and it compiles successfully, we celebrate:

If "Code compilation" is selected, the code has been run, and there are errors, the enabled "Ask AI Tutor" button is available:

On validated coding levels, there are options for "Code compilation", "Falling tests" and "I have a question":

If "Failing tests" is selected, and the tests have not yet been run, the student is prompted to run tests:

If "Failing tests" is selected, and the tests have run but there is a compilation error, the student is prompted to fix the compilation errors first (they can toggle to "Code compilation" and get the enabled "Ask AI Tutor" button"):

If "Failing tests" is selected, and the tests have run without compilation errors, but fail, they will see the "Ask AI Tutor" button. Note this button is currently always disabled because it doesn't yet return a meaningful response because I still need to pass the test files along with the student code to openAI. The AI just gets confused and times out right now.

If "Failing tests" is selected, and the tests have run without compilation errors, and succeed, celebrate:

Links

Testing story

Deployment strategy

Follow-up work

"Ask AI Tutor" for validation doesn't yet return a meaningful response because I still need to pass the test files along with the student code to openAI. Once that's solved in upcoming work, I'll enable the button.
Obviously we need to build out functionality for the General Chat option. A potential intermediate step just during this discovery phase could be to have a message like: "We're still training AI to be useful. You can't ask it anything right now, but if you could, what would you ask?" with a text input box that logs to Amplitude as a means of collecting data about what students want the AI to eventually be able to do. Stay tuned as discussions evolve!

Privacy

Security

Caching

PR Checklist:

Tests provide adequate coverage
Privacy and Security impacts have been assessed
Code is well-commented
New features are translatable or updates will not break translations
Relevant documentation has been added or updated
User impact is well-understood and desirable
Pull Request is labeled appropriately
Follow-up work items (including potential tech debt) are tracked and linked

…hat tutors

apps/src/code-studio/components/aiTutor/ai-tutor.module.scss

apps/src/code-studio/components/aiTutor/aiTutorPanel.jsx

Erin007 · 2023-11-17T03:00:39Z

apps/src/code-studio/components/aiTutor/validationTutor.jsx

+  const handleSend = async studentCode => {
+    dispatch(
+      askAITutor({systemPrompt: systemPrompt, studentCode: studentCode})
+    );


🤔 openAi also needs access to the test file(s)... find out where those live and how to send them along in the request

Are you planning to add that within the scope of this PR?

No - this was a note to myself. I'll disable the button for now and build out sending the test files along with the request in the next PR. Thanks for checking!

That work is happening here: #55076

apps/src/code-studio/components/aiTutor/validationTutor.jsx

…of help is selected

ebeastlake · 2023-11-17T23:30:56Z

apps/src/aiTutor/redux/aiTutorRedux.ts

@@ -14,24 +14,26 @@ const initialState: AITutorState = {
  isWaitingForAIResponse: false,
 };

+interface question {


nit: I think interfaces are meant to be capitalized?

I also wonder if a broader name for this is better, like ChatContext. (I think of "question" and "userMessage" -- a term used in Aichat for the next message the user sends to the endpoint -- as being equivalent.)

ebeastlake · 2023-11-17T23:36:10Z

apps/src/code-studio/components/aiTutor/aiTutorPanel.jsx

-  };
+const AITutorPanel = props => {
+  const dispatch = useDispatch();
+  const {level} = props;


nit: You could destructure this in props, i.e. const AITutorPanel = ({level}) => { ... }).

ebeastlake · 2023-11-17T23:40:29Z

apps/src/code-studio/components/aiTutor/aiTutorPanel.jsx

-        <img alt={i18n.aiBot()} src={icon} className={style.aiBotImg} />
-        {isCodingLevel && <CompilationAssistant levelId={level.id} />}
-      </AITutorPanelContainer>
+  const handleCheckboxChange = key => {


If only one can be checked, could you store the key associated with the checked box instead of the whole array of checkbox objects and their properties on state?

🤔 Every time a box is checked, I need to uncheck any previously checked boxes so I think I'd need state for the box to check as well as the ones to uncheck. It seemed simpler in my mind to just store all of them in state. What am I missing?

I struggled to enroll in the experiment locally so I couldn't test this, but here's the minor simplification I think would also work: https://github.com/code-dot-org/code-dot-org/pull/55082/files.

Though, to Molly's point, it looks like there's a RadioButton component available, which might better represent the intended mutual exclusivity: https://github.com/code-dot-org/code-dot-org/blob/staging/apps/src/componentLibrary/radioButton/README.md.

ebeastlake · 2023-11-17T23:43:01Z

apps/src/code-studio/components/aiTutor/compilationTutor.jsx

+
+// AI Tutor feature that explains to students why their code did not compile.
+const CompilationTutor = props => {
+  const levelId = props.levelId;


Same nit as above.

ebeastlake · 2023-11-17T23:44:11Z

apps/src/code-studio/components/aiTutor/compilationTutor.jsx

+      )}
+      {javalabState.hasRunOrTestedCode &&
+        !javalabEditorState.hasCompilationError && (
+          <h4>🎉 Your code is compiling successfully. Great work!</h4>


Was it a conscious choice not to start a translation file for these?

yes, it seems a little early since they aren't even really placeholders for string we know we'll need just part of my rogue self-organization scheme 😆 I'm not even sure that CSA is translated. But it's a good reminder for the kinds of work we'll need to leave time for as we near a more finalized notion of what AI Tutor will be and what strings will be involved.

ebeastlake · 2023-11-17T23:45:06Z

apps/src/code-studio/components/aiTutor/validationTutor.jsx

+
+// AI Tutor feature that explains to students why their code is not passing tests.
+const ValidationTutor = props => {
+  const levelId = props.levelId;


Same nit, last time 😄

ebeastlake · 2023-11-17T23:50:03Z

A few small comments, but nice work on getting a framework for all of these set up! 🎉

molly-moen · 2023-11-21T00:11:44Z

apps/src/code-studio/components/aiTutor/aiTutorPanel.jsx

+        {checkboxes.map(
+          cb =>
+            !cb.hidden && (
+              <Checkbox


if we do radio buttons instead of checkboxes that will enforce that only once can be selected right?

@molly-moen @ebeastlake Updated to Radio Buttons!

Screen.Recording.2023-11-21.at.11.48.34.AM.mov

molly-moen

Looks good, one redux suggestion!

molly-moen · 2023-11-21T17:20:19Z

apps/src/code-studio/components/aiTutor/compilationTutor.jsx

+// AI Tutor feature that explains to students why their code did not compile.
+const CompilationTutor = ({levelId}) => {
+  const dispatch = useDispatch();
+  const javalabState = useSelector(state => state.javalab);


Instead of getting all the state here I think we should get individual pieces as separate constants (for example, const hasRunOrTestedCode = useSelector(state => state.javalab.hasRunOrTestedCode);. This will prevent extra re-renders when something changes in the java lab state that we don't care about in this component.

ebeastlake · 2023-11-21T18:08:42Z

I like Molly's suggestion, but my feedback has been addressed! Nice work, Erin!

Erin007 added 2 commits November 16, 2023 19:32

set up select to switch between compilation, validation and general c…

2d5526e

…hat tutors

validationTutor asks AI Tutor

2b1c145

Erin007 changed the title ~~set up select to switch between compilation, validation and general c…~~ AI Tutor - validation v.1 in JavaLab Nov 17, 2023

Erin007 marked this pull request as draft November 17, 2023 02:55

Erin007 commented Nov 17, 2023

View reviewed changes

apps/src/code-studio/components/aiTutor/ai-tutor.module.scss Outdated Show resolved Hide resolved

Erin007 commented Nov 17, 2023

View reviewed changes

apps/src/code-studio/components/aiTutor/aiTutorPanel.jsx Outdated Show resolved Hide resolved

Erin007 commented Nov 17, 2023

View reviewed changes

apps/src/code-studio/components/aiTutor/validationTutor.jsx Outdated Show resolved Hide resolved

Base automatically changed from eb-ai-tutor-proto-1 to staging November 17, 2023 13:36

Erin007 added 3 commits November 17, 2023 08:59

merge staging, fix conflicts

79e83bd

Amplitude log for validation button, clear AI response when new type …

7238abd

…of help is selected

prompt to fix compilation errors before working on tests

8b1e15e

Erin007 changed the title ~~AI Tutor - validation v.1 in JavaLab~~ AI Tutor - set up how first 3 prototypes interact Nov 17, 2023

Erin007 requested review from ebeastlake and moneppo November 17, 2023 15:53

Erin007 marked this pull request as ready for review November 17, 2023 21:40

ebeastlake reviewed Nov 17, 2023

View reviewed changes

destructure props, question -> ChatContext

8d75b09

Erin007 requested review from ebeastlake, molly-moen and mikeharv November 20, 2023 21:55

molly-moen reviewed Nov 21, 2023

View reviewed changes

switch checkboxes for radio buttons

2fa0ba4

Erin007 requested a review from molly-moen November 21, 2023 16:52

molly-moen approved these changes Nov 21, 2023

View reviewed changes

separate pieces of state to avoid unnecessary re-renders

250f304

Erin007 merged commit 0f7a9b7 into staging Nov 22, 2023
2 checks passed

Erin007 deleted the eb-ai-tutor-validation-proto branch November 22, 2023 16:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AI Tutor - set up how first 3 prototypes interact #55015

AI Tutor - set up how first 3 prototypes interact #55015

Erin007 commented Nov 17, 2023 •

edited

Erin007 Nov 17, 2023

ebeastlake Nov 17, 2023

Erin007 Nov 20, 2023

Erin007 Nov 22, 2023

ebeastlake Nov 17, 2023 •

edited

ebeastlake Nov 17, 2023

ebeastlake Nov 17, 2023

ebeastlake Nov 17, 2023

Erin007 Nov 20, 2023

ebeastlake Nov 21, 2023 •

edited

ebeastlake Nov 17, 2023

ebeastlake Nov 17, 2023

Erin007 Nov 20, 2023

ebeastlake Nov 17, 2023

ebeastlake commented Nov 17, 2023

molly-moen Nov 21, 2023

Erin007 Nov 21, 2023

molly-moen left a comment

molly-moen Nov 21, 2023

ebeastlake commented Nov 21, 2023

AI Tutor - set up how first 3 prototypes interact #55015

AI Tutor - set up how first 3 prototypes interact #55015

Conversation

Erin007 commented Nov 17, 2023 • edited

Links

Testing story

Deployment strategy

Follow-up work

Privacy

Security

Caching

PR Checklist:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebeastlake Nov 17, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebeastlake Nov 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebeastlake commented Nov 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

molly-moen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebeastlake commented Nov 21, 2023

Erin007 commented Nov 17, 2023 •

edited

ebeastlake Nov 17, 2023 •

edited

ebeastlake Nov 21, 2023 •

edited