"Bizarro" Devin Brainstorming Features

Core behaviors

• auto write code into sketch.js (incrementally)
• look at errors and fix bugs
• narrate its process of writing the code
• preview pane to show p5.js canvas (w/ auto-refresh)

Interaction Features

• Use whisper to listen to what I am saying
• Narrate the process, banter with me using TTS model?

ML Model

• Fine-tune a model?
• RAG with nature of code book or other CT challenge examples?
• multi-modal so it can look at the canvas

Models/frameworks under consideration

My preference is for on device and fine-tuned model.

• Gemma
• transformers.js models
• llama (via ollama)
• localAI models