Backend upgrade: Stability & Firestore, Pinecone, Silero VAD & Docs by skywinder · Pull Request #1981 · BasedHardware/omi

skywinder · 2025-03-12T00:46:55Z

🚀 Backend Bootstrap Fixes & Improvements

This PR includes multiple fixes for backend initialization and configuration.
Each fix is in a separate commit for easier review.

🔧 Fixes & Enhancements

Fix Firestore client initialization with explicit project ID
4b30d81
Improve Pinecone vector database initialization with proper configuration handling
763c7bc
Add project ID handling for Google Cloud Storage client initialization
cbe4a95
Enhance GitHub API token handling and update documentation
715e091
Improve Silero VAD model loading with error handling and fallback
dd659bc
docs: Add troubleshooting section for Opus library issues
b8a5a3b
feat: Add Typesense mock client for development & improve configuration handling
5cdbd46

📖 Documentation Updates

docs: Enhance Typesense configuration guide in README
dc0ee8c

🎤 Audio Processing Improvements

feat: Fix Silero VAD model download & loading mechanism
34650a8
fix: Update Silero VAD model download URL & simplify file retrieval
3ab94bd
feat: Add PyOgg error handling & fallback mechanism for Opus codec
ffd5e69

🐳 Docker Setup Now Works Out of the Box!

Docker & Docker Compose setup added!
2d7942c
da4dbbb

🚀 How to Run

Local

uvicorn main:app --reload --env-file .env

Docker

docker compose up -d

- Update README.md with detailed instructions for setting up GitHub token - Improve GitHub API request handling in firmware and plugins utils - Add warning messages for missing GitHub token - Handle rate limit scenarios - Provide clearer guidance on optional GitHub token configuration

…hanism - Add SSL certificate verification bypass - Implement try-except block for model loading - Create mock VAD functions for graceful degradation - Handle potential model loading failures

- Include detailed steps for resolving Opus library installation issues - Add verification step to confirm successful library import

…dling - Implement conditional Typesense client initialization - Create mock Typesense client for development when configuration is missing - Update README.md with Typesense configuration instructions - Add warning message for unconfigured Typesense setup

- Provide setup instructions for Typesense Cloud and self-hosted environments - Clarify host, port, and API key configuration steps

- Implement direct model file download from GitHub repository - Add fallback mechanism for Silero VAD model loading - Handle potential download and loading errors

- Update model file download URL to latest repository location - Remove redundant file downloads for utils and example files - Streamline model file retrieval process

- handling of PyOgg import errors - Create mock OpusDecoder class for limited functionality - Update README with troubleshooting steps for PyOgg library

skywinder · 2025-03-12T03:00:30Z

Hooray! I did it! 🎉 Finally, it’s up and running! 🙌

Hope this helps many others!

@mdmohsin7 @thinhx I’d really appreciate a quick review. It has passed all the tests and checks. I managed to overcome 20+ errors, and it’s at least running for now. However, this is my first launch, so please take a look. Thanks! 🙏

neooriginal · 2025-03-12T03:16:19Z

Reviewing right now...

neooriginal · 2025-03-12T03:28:15Z

lgtm, starts without issues, but ran into a windows-specific problem.
needs further review from someone else.

skywinder · 2025-03-12T03:31:25Z

lgtm, starts without issues, but ran into a windows-specific problem. needs further review from someone else.

I didn’t test it on Windows, to be honest—only on Mac. I just added the obvious path from the manuals, just in case. Better this than nothing!

…her service to the max capacity

- Removed Google Cloud credentials configuration from docker-compose.yml - Updated README and Backend_Setup.mdx to reflect changes in credentials handling and installation instructions

skywinder · 2025-03-12T18:49:50Z

Hooray! Docker and Docker Compose are now working! Added files and the dock! Enjoy! 😉

2d7942c
da4dbbb

Changed exposed port from 8080 to 8000 in Dockerfile and docker-compose.yml

mdmohsin7 · 2025-03-13T07:39:43Z

backend/Dockerfile


 COPY --from=builder /opt/venv /opt/venv
-COPY backend/ .
+COPY . .


I think this will break the gcp deployment workflow (not really sure). @skywinder can you pls once cross check the dockerfile.datadog file and also the gcp_backend.yml file in .github/workflows (if you haven't already)

GCP deployment workflow specifically uses Dockerfile.datadog, not the Dockerfile.
so the immediate changes to your regular Dockerfile won't directly break the GCP workflow.

So, gcp_backend.yml refer to dockerfile.datadog, which is not affected.

omi/.github/workflows/gcp_backend.yml

Lines 65 to 66 in 3eb8df8

file: ./backend/Dockerfile.datadog

push: true

It has 2 differences

Path: in Dockerfile vs Dockerfile.datadog: - since we are running it from /backend folder directly
Otherwise, we’ll have to move this Dockerfile to ../ to make it work, but I think the proper location is in the backend folder.

Port: I set 8000 from backend readme and docs.

omi/backend/README.md

Lines 86 to 87 in 43508d6

ngrok http --domain=example.ngrok-free.app 8000

```

I see that in Dockerfile.datadog it's 8080. Should I set it to 8080 in dev setup as well?

skywinder · 2025-03-14T00:03:05Z

TL;DR:
• Silero VAD: Fixed broken distribution, updated download link, specified versions, and added error handling for model loading.
• Docker: Corrected invalid paths.
• Typesense: Added mock client to enable running without a configured service.
• Logging & Error Handling: Implemented logs and error reporting for all services (Firestore, GCS, Silero VAD, PyOgg); the system now reports issues clearly instead of failing silently.

…ADME

beastoin · 2025-03-15T01:49:08Z

guys, feel free to discuss this one btw my tips are:
1/ run the back-end yourself with the current configs and requirements successfully.
2/ simplify setup steps by separating the required envs(should be minimal) and the optional vars. then make the optional vars truly optional, meaning if developers don't add theses optional vars, the back-end should still run and core features work well.
3/ automate back-end setup with docker(or any containerization) and scripting
4/ update the document regarding the changes.

keep the PR minimal and focused on single responsibility.

getting this repo up and running is not that hard, but if it took too long for you, dm me discord / @thinh

@skywinder @mdmohsin7

beastoin · 2025-03-17T04:52:51Z

ping me after 2 weeks (or when you are ready for changes)

/ draft

@skywinder

beastoin · 2025-03-24T23:17:51Z

/ closed

feel free to reopen it any time man.

keys

… and admin marketplace

beastoin · 2025-04-14T10:52:50Z

hey man, it's cleaning time.

/ closed

3 days with no updates
feel free to reopen it anytime

skywinder added 11 commits March 12, 2025 01:43

Fix Firestore client initialization with explicit project ID

4b30d81

Pinecone vector database initialization with configuration handling

763c7bc

Add project ID handling for Google Cloud Storage client initialization

cbe4a95

Improve Silero VAD model loading with error handling and fallback mec…

dd659bc

…hanism - Add SSL certificate verification bypass - Implement try-except block for model loading - Create mock VAD functions for graceful degradation - Handle potential model loading failures

doc: Add Opus library troubleshooting

b8a5a3b

- Include detailed steps for resolving Opus library installation issues - Add verification step to confirm successful library import

docs: Enhance Typesense configuration guide in README

dc0ee8c

- Provide setup instructions for Typesense Cloud and self-hosted environments - Clarify host, port, and API key configuration steps

feat: Improve Silero VAD model download and loading mechanism

34650a8

- Implement direct model file download from GitHub repository - Add fallback mechanism for Silero VAD model loading - Handle potential download and loading errors

fix: Update Silero VAD model download URL and simplify file retrieval

3ab94bd

- Update model file download URL to latest repository location - Remove redundant file downloads for utils and example files - Streamline model file retrieval process

feat: Add PyOgg error handling and fallback mechanism for Opus codec

ffd5e69

- handling of PyOgg import errors - Create mock OpusDecoder class for limited functionality - Update README with troubleshooting steps for PyOgg library

skywinder marked this pull request as ready for review March 12, 2025 03:00

Merge branch 'main' into backend-improments

c36bdb7

skywinder force-pushed the backend-improments branch from ce5684e to c36bdb7 Compare March 12, 2025 03:22

skywinder changed the title ~~Backend-improments~~ Backend upgrade: Stability & Firestore, Pinecone, Silero VAD & Docs Mar 12, 2025

neooriginal and others added 5 commits March 12, 2025 13:28

Fixed App Store Icon in ReadMe

0710acd

Removing the VAD on external trigger for audio-bytes; pushing the pus…

e368cc6

…her service to the max capacity

x2 connections for the external integration

8af4897

refactor: Restructure app list page layout and category rendering

88215d1

feat: Improve Google Cloud credentials handling in backend

1082646

This was referenced Mar 12, 2025

Link to Silero VAD model is invalid (404) OwlAIProject/Owl#58

Open

Fix PyOgg Import Issue: `name 'c_int_p' / ‘c_uchar_p’ is not defined #1988

Closed

PyOgg Import Issue: name 'c_int_p' / 'c_uchar_p' is not defined #1989

Closed

skywinder added 2 commits March 12, 2025 19:33

add docker compose

2d7942c

chore: Update backend Docker configuration and documentation

da4dbbb

- Removed Google Cloud credentials configuration from docker-compose.yml - Updated README and Backend_Setup.mdx to reflect changes in credentials handling and installation instructions

Update Docker configuration to use port 8000

3eb8df8

Changed exposed port from 8080 to 8000 in Dockerfile and docker-compose.yml

This was referenced Mar 12, 2025

rename friend_private to omi_app #1987

Merged

windows backend compatibility #1983

Closed

mdmohsin7 reviewed Mar 13, 2025

View reviewed changes

doc: Add GOOGLE_CLOUD_PROJECT variable to .env.template and update RE…

77d7ad3

…ADME

skywinder added 2 commits March 16, 2025 22:16

Merge branch 'main' into backend-improments

80fe051

gitignore

1d2ac3f

beastoin marked this pull request as draft March 17, 2025 04:52

beastoin closed this Mar 24, 2025

skywinder reopened this Apr 10, 2025

skywinder added 3 commits April 10, 2025 03:04

Merge branch 'main' into backend-improments

d22fa0f

clenap optional variables and wrap try catch to make it work without

7e9c238

keys

update .env.template remove unused var and add mised bucket variables…

6d2cc7c

… and admin marketplace

beastoin closed this Apr 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backend upgrade: Stability & Firestore, Pinecone, Silero VAD & Docs#1981

Backend upgrade: Stability & Firestore, Pinecone, Silero VAD & Docs#1981
skywinder wants to merge 26 commits intoBasedHardware:mainfrom
skywinder:backend-improments

skywinder commented Mar 12, 2025 •

edited

Loading

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

neooriginal commented Mar 12, 2025

Uh oh!

neooriginal commented Mar 12, 2025

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

mdmohsin7 Mar 13, 2025

Uh oh!

skywinder Mar 13, 2025 •

edited

Loading

Uh oh!

skywinder commented Mar 14, 2025

Uh oh!

beastoin commented Mar 15, 2025

Uh oh!

beastoin commented Mar 17, 2025

Uh oh!

beastoin commented Mar 24, 2025

Uh oh!

beastoin commented Apr 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

skywinder commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Backend Bootstrap Fixes & Improvements

🔧 Fixes & Enhancements

📖 Documentation Updates

🎤 Audio Processing Improvements

🐳 Docker Setup Now Works Out of the Box!

🚀 How to Run

Local

Docker

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

neooriginal commented Mar 12, 2025

Uh oh!

neooriginal commented Mar 12, 2025

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

skywinder commented Mar 12, 2025

Uh oh!

mdmohsin7 Mar 13, 2025

Choose a reason for hiding this comment

Uh oh!

skywinder Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

skywinder commented Mar 14, 2025

Uh oh!

beastoin commented Mar 15, 2025

Uh oh!

beastoin commented Mar 17, 2025

Uh oh!

beastoin commented Mar 24, 2025

Uh oh!

beastoin commented Apr 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

skywinder commented Mar 12, 2025 •

edited

Loading

skywinder Mar 13, 2025 •

edited

Loading