SimpleScribe v1.3.0 - now connects to OpenAI (bring your own key)

SimpleScribe v1.3.0

Made available for testing and feedback. Not endorsed at this time for clinical care.

Prior versions were focused on offline transcription and making it easier to copy everything you needed to an LLM of your choice. This version uses OpenAI directly in the program. Transcription remains local and uses Whisper.cpp,

PLEASE NOTE: There is no key to OpenAI's API included here, but a placeholder .env file is available for you to fill in.

UI has been been updated. This release includes both ggml-small.en.bin and ggml-tiny.en.bin. If you want to trial using the tiny model (faster but less accurate), open the config file, change the name from ggml-small.en.bin to ggml-tiny.en.bin, and restart.

Hope you enjoy trialing this.

Instructions:

1. Make sure you have Python downloaded to your computer and it's in your PATH (check box during installation)
2. Downlowd the v1_3_0_SimplescribeVA.zip below from Assets to your OneDrive desktop.
3. Unzip the folder and extract the contents. You should see a folder, a Setup shortcut and a Start SimpleScribe shortcut.
4. Open the .env file (if you don't see it, change your view so you can see file extensions), put your OpenAI API key in the placeholder, and save.
5. Double click Setup. This may take a few minutes, and you may need to repeat on whatever computer you're on.
6. Double click Start SimpleScribe shortcut whenever you'd like to use
7. Please note, this version does not yet prevent you from starting another session at a different computer. Make sure to close the program (terminals) or at least stop recording when you move to a new computer.

Real-time with Windows Speech Recognition

REPLACED WHISPER WITH WINDOWS SPEECH RECOGNITION

The whole thing is simplified now. Just the Flask backend (run_local_server.py) that now starts and stops a small C# console app that uses the Windows Speech Recognition (WSR) engine built into Windows. The accuracy is terrible, especially for clinical terms. However, it seems most LLMs can figure out much of what was likely actually. This release it to trial the trade-offs between real-time functioning and initial transcription accuracy. The default prompts have the LLM first attempt to correct the transcript, then have the user verify before proceeding to note generation.

Instructions:

1. Have Python installed and included in your PATH
2. Open the zipped file and move the SimpleScribeVA folder to your desktop
3. Click Setup.bat. Process needed the first time only on each of your computers
4. Click StartSimpleScribeVA.bat to get rolling

The included changelog and readme are incorrect. Windows Speech Recognition is on the device, so it remains an offline option, at least for this transcription portion

SimpleScribeVA v1.1.1

SimpleScribeVA v1.1.1

Warning: This offline virtual medical scribe, intended to work with the secure, HIPAA-compliant LLM of your choice, remains unapproved by the VA. I'm releasing it to allow for testing and feedback.

Background: Virtual medical scribes are rapidly becoming ubiquitous across health care, with both significant potential benefit as well as risk. The benefit of a VA-specific scribe is that it would allow for customization of prompts for professions that are currently underserved in the market (e.g. social workers and nurses), improved focus on Veteran-specific issues and programs like HUD-VASH, and eventual integration with VISTA/CPRS. Most importantly, careful internal testing would allow for rapid feedback and correction on any issues of concern that might arise.

Instructions: First make sure you have downloaded Python 3.13, and that during installation you checked the box to have it added to the Path for all users. Second, download this zip to your OneDrive desktop on a Windows computer and double-click to decompress. Double click on Setup.bat. This may take several minutes to finish, and you may need to repeat the Setup.bat process on new computers. Then double-click StartSimpleScribeVA.bat any time you'd like to get started. A browser will open to http://127.0.0.1:5000 and in a few moments you should see the SimpleScribeVA interface.

CHANGES IN THIS RELEASE
2025-04-22: v1.1.1: Minor bug fixes and updated the design. The bugs were that the notice of "Transcribing..." would not turn off even after transcription ended, and the transcripts were not displaying correctly. Minor improvements in the default prompts. Minor updates in styling.

SimpleScribeVA v1.1.0 (transcription model included)

SimpleScribeVA v1.1.0

Warning: This offline virtual medical scribe, intended to work with the secure, HIPAA-compliant LLM of your choice, remains unapproved by the VA. I'm releasing it to allow for testing and feedback.

Background: Virtual medical scribes are rapidly becoming ubiquitous across health care, with both significant potential benefit as well as risk. The benefit of a VA-specific scribe is that it would allow for customization of prompts for professions that are currently underserved in the market (e.g. social workers and nurses), improved focus on Veteran-specific issues and programs like HUD-VASH, and eventual integration with VISTA/CPRS. Most importantly, careful internal testing would allow for rapid feedback and correction on any issues of concern that might arise.

Instructions: First make sure you have downloaded Python 3.13, and that during installation you checked the box to have it added to the Path for all users. Second, download this zip to your OneDrive desktop on a Windows computer and double-click to decompress. Double click on Setup.bat. This may take several minutes to finish, and you may need to repeat the Setup.bat process on new computers. Then double-click StartSimpleScribeVA.bat any time you'd like to get started. A browser will open to http://127.0.0.1:5000 and in a few moments you should see the SimpleScribeVA interface.

CHANGES IN THIS RELEASE

Improved Accuracy: This release uses a larger, more accurate Whisper transcription model (ggml-medium.en.bin as opposed to ggml-small.en.bin), without a significant delay in transcription time. Transcription remains entirely on your computer.

More User Feedback: This releases asks for JSON instead of TXT files from Whisper, and uses the metadata to provide confidence scores for tokens (words or syllables). Those below a certain confidence level are highlighted in the transcription box, so the user can quickly fix glaring issues right in the transcript box before copying to the clipboard. The transcript also strips out some disfluencies like um or ah, and periods of silence.

Default Prompts Improved: The default prompts now specify that you're caring for Veterans to allow for Veteran specific services like VASH to be incorporated more easily. More importantly, the prompts now ask GPT to slow down and take the additional sequential steps of attempting to 1) correct obvious transcription errors based on context 2) categorize statements (e.g. S, O, A, P) and 3) ask the user to approve the corrected, categorized statements before creating the note. This pre-categorization reduces hallucinations in the final product, and the incorporation of an additional user check improves overall reliability.

SimpleScribeVA Offline v1.0 with model included

SimpleScribeVA (Offline Version)

SimpleScribeVA is a privacy-focused, offline-first tool for virtual medical scribing. This version is intended for use with the secure LLM of your choosing, such as VA GPT beta.

It allows clinicians to transcribe audio (dictations or patient visits) entirely on their computer using Whisper.cpp, and structure notes using prompt templates like SOAP or H&P.

This offline version does not require an OpenAI API key — instead of generating notes automatically, users can copy a prompt of your choosing + transcript content (either a dictation or a patient visit) to paste into the VA GPT Beta interface.

This release includes the Whisper model you will need to transcribe, which is why it is a large file. The model used is Whisper.cpp's ggml-small.en.bin transcription model, taken from https://huggingface.co/ggerganov/whisper.cpp/blob/main/ggml-small.en.bin. If you decide to use a different model, please update the file name in config.json.

⚠️ AS OF THIS WRITING ON 4/16/25, THIS SOFTWARE IS NOT OFFICIALLY ENDORSED BY THE VA. IN ADDITION, PLEASE REVIEW THE INCLUDED LICENSE. IN PARTICULAR, THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

---

✨ What This Version Does

• Requires that you obtain verbal consent from anyone being recorded
• Records audio from your default microphone in 30-second overlapping chunks
• Transcribes audio using Whisper.cpp (entirely offline)
• Audio is immediately deleted after transcription
• Lets you paste in optional chart data
• Lets you select and preview a documentation prompt and save your own custom prompts
• Allows you to copy a prompt + transcript + chart data for pasting into the VA GPT Beta
• Saves transcripts and templates in a folder that can sync with OneDrive across secure devices
• Prior transcripts can be reviewed and deleted from within the program
• Future: Potentially connect directly to VA GPT beta and/or clinical data

---

♻ Installation & Setup (Windows)

Step 1: Install Python 3.13 (if not already on your computer)

• Go to: https://www.python.org/downloads/
• Download Python 3.13 (64-bit)
• During installation, check the box that says:
  ✅ "Add Python to PATH" (very important)

Step 2: Download SimpleScribeVA to OneDrive

• Download this release (SimpleScribeVA_Offline_v1.0_with_model_included.zip) file and double click to open it. Move the folder SimpleScribeVA_Offline to your Desktop. Keeping it on your OneDrive Desktop ensures your transcripts and custom templates sync across devices.

Step 3: First-Time Setup

• Double-click Setup.bat in the folder
  • This creates a virtual environment and installs required packages
  • May take a few minutes

Step 4: Daily Use

• After setup, launch the app anytime by double-clicking:
  StartSimpleScribeVA.bat
  (Consider making a shortcut to this on your desktop)
• A browser window will open to:
  http://127.0.0.1:5000
• Click "Verify and Continue" on the landing page
• Close the terminal/command prompt windows (black, text-filled boxes) to close the program fully

💡 You can bookmark that local address (http://127.0.0.1:5000) for easy access, as long as the app is running.

---

🔄 Workflow

1. Click Start Recording to begin recording audio (button glows subtly with your voice)
2. Speak normally — every ~30 seconds, the app saves and transcribes a chunk
3. Click Stop Recording. Once you stop recording, the final audio will finish transcribing
4. You can start and stop recording multiple times during a session
5. Select a prompt template from the dropdown
6. Preview the prompt in the box below
7. Click "Copy Prompt / Transcript / Chart Data"
8. Paste into the VA GPT Beta (https://vagptbeta.va.gov) interface to generate a note
9. Save your transcript by clicking End Session

---

🌐 Folder Structure

SimpleScribeVA/
├── run_local_server.py         # Flask app (backend)
├── monitor_transcription.py    # Whisper audio chunk processor
├── record_audio.py             # Mic recording logic
├── chunks/                     # Audio files are saved here until transcribed, then deleted. The folder will be created the first time you run SimpleScribeVA
├── static/                     # app.js which contains the various frontend javascript functions
├── templates/                  # html files for pages
│   ├── default/                # Built-in prompt templates
│   └── custom/                 # User-created templates
├── transcripts/                # Saved transcripts by date. The folder will be created the first time you run SimpleScribeVA 
├── whispercpp/                 # Local Whisper binary + model
├── .env                        # Where API keys for direct connection to GPT would go
├── live_transcript.txt         # Where the live transcript is kept during a session
├── config.json                 # Tracks model name (in this case Whisper's ggml-small.en.bin)
├── Setup.bat                   # One-time environment setup
└── StartSimpleScribeVA.bat     # Launcher

---

🤝 How It Works (Under the Hood)

• The Flask app (run_local_server.py) runs your local web interface
• Audio is recorded live via sounddevice and saved to chunks/
• monitor_transcription.py watches chunks/, runs Whisper.cpp on new audio, and appends results to live_transcript.txt
• You can copy the transcript + chart data + prompt directly to your clipboard
• Eventually, a future version may support direct GPT-4/VA API integration

---

📚 Requirements.txt Review

The current version requires:
flask
python-dotenv
sounddevice
scipy
openai>=1.0.0
psutil
Note:
• openai is included for future compatibility but is unused in this version
• scipy is only needed by some sounddevice builds

---

⚠️ Notes

• Transcripts are not encrypted, so they should be stored in your secure VA OneDrive account
  (This will occur automatically if the folder lives on your OneDrive desktop)
• Transcripts can be reviewed and deleted from the archive page (click the folder icon in the top-right corner of the app)

---

🏠 Designed For

• Clinicians who want to draft notes efficiently using GPT-based tools
• Settings where network connectivity or privacy are concerns
• VA providers testing GPT Beta workflows

---

Simha.Reddy@va.gov
April 2025