💫 Use AI to automatically fix syntax errors

[fonsp]

Whenever you hit a Syntax error in Pluto, we will offer a button to fix it automatically, using Claude 3.5 haiku.

Schermopname.2025-04-24.om.13.20.52.mov

It also includes a prompt to explain what will happen (not shown on the video), asking for permission.

First AI feature

This is Pluto's first AI feature! I chose this one over #2517 because it has a small, clear scope, and it is easy for the LLM to be successful and helpful.

I was hesitant to integrate LLM features in the past because of the usual reasons. But I think that it is undeniable that LLMs can be hugely beneficial to our core mission: "to make scientific computing more accessible and fun".

Future

I'm interested to implement more LLM features in the future :)

I'm also thinking of bring-your-own-chatbot features, where Pluto does not integrate with an LLM directly, but it just makes it easier to use alongside an existing program like ChatGPT or Cursor. Pluto could generate a good prompt for you (Pluto knows the relevant context of a cell) and copy it to your clipboard. All you need to do then is paste it in your chatbot.

AI model

We use Claude 3.5 haiku 2024 through the anthropic API. It's a cheap model that works well enough for this use case.

The API is called through our backend, which is closed-source (might open source in the future). The backend is implemented in JS with Deno Deploy. Pluto devs can contribute to the source code here: https://github.com/JuliaPluto/pluto-simple-llm-features

Cost

This application is cheap enough that we can offer it for free to all users without requiring a login (probably). I set up a spending limit. We can take the service offline at any point, and the button will simply not show up anymore.

I think we will stay within the Deno deploy free limits.

Configuration

You can use the setting enable_ai_editor_features for this. To turn it off:

Pluto.run(enable_ai_editor_features=false)

The feature will be disabled automatically if Pluto detects that ChatGPT is blocked on your network: #3211

[digital-carver]

It would be great if there was a way to either:

• go back and forth between the pre-AI-fix version of the code and the fixed code, or
• get a preview of the AI suggestion and be able to either accept it or reject it

Right now, it's hard to see what the AI is changing, especially because there's an unpredictable amount of delay between pressing the button and the change happening. Implementing either of the above would give us a way to compare the two versions, and especially for students/learners, be able to learn from it.

(I can make an issue for this if you think that'll be helpful, let me know if so!)

[fonsp]

Hi @digital-carver ,

That's a nice idea! Feel free to repost in a new issue.

What did you think of the feature besides this?

[ctessum]

This looks really interesting! One thing, though, is that I use Pluto for a class that I teach, both for in-class exercises and homeworks (with internet connectivity enabled) and also for exams (with internet connectivity disabled). So I would worry with this feature that students would become reliant on the AI during their homeworks and then not be able to fix any bugs during the exams.

For this reason, would it make sense to also have an option where the AI could just give hints about what is wrong, but still require the user to fix the code themselves?

[fonsp]

Hi @ctessum ! Thanks for your feedback!

Just FYI, right now this is only for syntax errors (code does not parse as legal Julia, like misspelling function), not for any other type of error.

For syntax errors, Julia already gives a couple of hints of what is wrong, right? In the video, the hints you see are from Julia, and with AI disabled you can already see those hints. But your suggestion is to make the hints better using AI, right? (Maybe use more words, less technical?)

---

I also like your general suggestion (and from @digital-carver as well) about the educational opportunity here: to show what was wrong and what got fixed, and to not just fix it magically.

[ctessum]

Yep, that would be the idea. I spend quite a bit of time in the class working with the students to interpret their error messages, and I guess AI could do that too.

Or, another possibility could be to do a "suggested changes" type of thing that shows the student the change that should be made (maybe with highlighting or color coding), but doesn't actually make the change. Just something that elicits more of a "hey I can do this" type of response and less of a "wow that was magic" type of response.

But I think this is really cool too!

[digital-carver]

What did you think of the feature besides this?

Definitely a very neat idea, especially useful for beginners since syntax errors like this are the most frustrating part of the learning process. This would be massively helpful in being able to explore the language concepts without getting stuck at every syntax hurdle.