Let's Learn Traditional Dance! (Final Project: Orbit Future Academy)

Oh Dance offers a real and unique traditional dance learning experience with the support of Artificial Intelligence (Machine Learning). It is said to be real because we will learn to directly demonstrate the basic dance movements and AI will predict and classify the movements we imitate. In addition, the search recommendation feature makes it easier for us to find traditional dances based on keywords, regional names, clothing, accessories, and musical instruments.

Detection and Classification of Basic Dance Movements (CV)

Datasets

The dataset used in the training model was obtained from the studio (sanggar) where one of our team members practiced. Each traditional dance has a dataset of 60 images which are evenly divided into three classes of basic dance moves.

Models

Models for prediction (PoseNet with Fine Tuning) and classification (MobileNet with Fine Tuning) of basic dance movements were built using the help of Teachable Machine - Pose (Google's AutoML Platform). For the implementation using the Teachable Machine Library - Pose. Note: The model that has been created is not included in this repo because it is in the development stage.

Result and Note

Accuracy for Classification Models (Pakarena Dance Movement)

Loss for Classification Model (Pakarena Dance Movement)

Note:

Graphics accuracy looks very good even almost 100% accuracy. However, this happens because of the large number of biases caused by the minimal variation in the dataset used (imbalanced dataset). This also applies to the loss model graph showing a very low predictive error value because the training and testing data have almost identical variations. In addition, currently the dataset used is still being developed (currently 60 images per traditional dance).

Deploy (Using Teachable Machine Library, Tensorflow Js and P5 Js with Float Model)

https://oh-dance.azurewebsites.net/learn-dances

Search and Recommendation Systems (NLP)

Installaton

Use pip to install prerequisites on requirements.txt, if you on google colab you can skip this process.

pip install -r requirements.txt

Usage

Follow the cell on the notebook research file

Datasets

The dataset consists of 5 traditional dance classes with 100 lines of sentences per class. Each line contains keywords, regional names, fashion, accessories, musical instruments and others. Dataset collection method using Google Search Engine. Source: Dataset NLP by Oh Dance

Models

The model was built using the Long Short-Term Memory (LSTM) algorithm and improved by adding a dropout layer to reduce overfitting in the model.

Result

The accuracy of the training dataset is 81% while the testing dataset is 84%.

Deploy (Using Tensorflow Js with Quant Model)

Click search menu: https://oh-dance.azurewebsites.net/