Skip to content

Signlytic AI v1.1.0 - Live recognition and sentence assembly

Choose a tag to compare

View all tags
@Iyanuoluwa007 Iyanuoluwa007 released this 12 Aug 12:29
· 69 commits to main since this release
v1.1.0
f7b4e36

The Signlytic AI application: the local FastAPI dashboard that runs the full translation pipeline on your own GPU, with live BSL recognition, sentence assembly and speech output.

This is the release to track for the main app. Newer versions will follow here.

Running it

conda activate BSL
cd <repo>
python app_server.py

Then open http://localhost:8000

app.py is the combined Gradio application; app_server.py is the FastAPI
dashboard that serves the live view and the API.

New in v1.1.0

Live recognition view. Recognised glosses accumulate as you sign, commit
with a fade once settled, and show a confidence readout for the word being
recognised.

Sentence assembly. /api/live/assemble takes the accumulated glosses and
returns a natural English sentence, with speech output.

Emotion detection. Face-api runs alongside recognition and maps
expression onto the signing view.

Faster first frame. Models are warmed on startup, so the first live frame
no longer pays the model load cost.

Cerebras primary for gloss to English. Assembly now tries Cerebras
gpt-oss-120b first and falls back to Groq llama-3.3-70b-versatile on error,
timeout, or a missing key, with a short timeout so a hang cannot stall the
pipeline. Measured at 450-720 ms per sentence on the primary path, and all
three fallback paths verified. Key values are never logged.

API

GET  /api/health              backend and model status
POST /api/d1/video            BSL video to English
POST /api/d1/glosses          BSL glosses to English
POST /api/d2/text             English text to BSL glosses
POST /api/d2/audio            speech to BSL glosses
POST /api/live/frame          one live frame, returns recognised gloss
POST /api/live/assemble       accumulated glosses to sentence and speech
GET  /api/signs/{gloss}       pose frames for a gloss
GET  /api/avatar/{name}       avatar model

Requirements

  • An NVIDIA GPU for recognition and speech synthesis
  • The BSL conda environment
  • GROQ_API_KEY, and CEREBRAS_API_KEY for the primary assembly path

Related releases

Known limitations

  • In 3D mode the avatar's hands sit lower than they should. 2D is unaffected.
  • Recognition and speech need a local GPU; the hosted site runs text
    translation only.
Assets 2
Loading