$$\color{#a1585c}SignWise$$

App with a focus of gamification of learning an American Sign Language, prioritizing engagement and motivation 😊

View Demo · Report Bug · Request Feature

🌟 About The Project

With the increasing availability of technology, the number of available applications on the market is also growing. Despite this, there are currently relatively few applications supporting sign language learning. This led to the idea of ​​developing such software, inspired by well-known apps like Duolingo, which motivate users to learn through experience points, user progress level, and various learning modes. The project aims to increase the accessibility of sign language learning options and to influence motivation to practice through the use of gamification elements.

SignWise is an AI-powered educational app. The program we developed enables real-time feedback by implementing a classification model that utilizes a camera to determine whether the displayed sign is correct.

(back to top)

👾 Built With

• 
• 
• 

(back to top)

🎯 Features

1. 🤴🏻 User account

- saved user progress (experience points -> level progression)
- ability to change username
- ability to reset experience progress

2. 📚 Learning mode

- scrolling list of alphabet or numbers that allows users to explore ASL signs before testing themselves,
  making it easier for beginners to understand and revisit how individual signs are performed

3. FOUR testing modes where you can get experience points:

a) 🕹️ Quiz

    - 3 categories to choose from: alphabet, numbers & mixed mode
    - sets of 10 questions with 4 answer options each
    - set completion pop-up displaying the score and options to replay with a new set or return to the main menu

b) 🃏 Flashcards

    - 3 categories to choose from: alphabet, numbers & personal favourites - you decide which flashcards make the list
    - smooth animations (stack animation, swiping and flipping) make learning more dynamic and enjoyable
    - finish pop-up with learning statistics ("known" vs "still learning" flashcards) and navigation options:
        - back to the menu
        - "still learning" flashcards repetition
        - whole set repetition

c) 🧩 Match Game

    - 3 categories to choose from: alphabet, numbers & mixed mode
    - time-based gameplay - beat your personal best!
    - smooth matching animations for better user experience
    - set completion pop-up showing completion time with options to play again using a new set or navigate back to the main menu

d) 🤖 AI Camera

    - real-time ASL sign recognition using the device camera
    - AI Quiz:
        - the app prompts the user to perform a specific sign
        - live gesture analysis with instant correctness feedback
        - current prediction hint highlighting incorrect hand position or movement

See the open issues for a full list of proposed features (and known issues).

(back to top)

🛠️ Future enhancements

• difficulty levels in quiz & match game
• connecting the rest of the features with Room Database
• expanding set database -> adding new categories such as greetings, cooking, traveling etc. including popular phrases like "Hello", "Thank you"
• enhancing AI model -> e.g. adding Hand Landmarker for better hand recognition
• daily streaks
• achievements

(back to top)

Project Lead Team ❤️