Capstone design project for graduation
This is a repository for the capstone design project that allow users to identify their sleep positions by classifying the sleep position image data received by a deep learning model, recording them on the app screen, and generating statistics.
In this work, ResNet50 was trained by Google Colab. We purchased a home CCTV and recorded sleeping videos. Plus, we applied data transformation, data augmentation and various improvement techniques improving overfitting to increase the test accuracy to over 90%. The ResNet50 was converted to 'tflite' format and saved in Android Studio. Then, we created an application so that users can easily identify their sleep positions. The application is divided into MAIN, INFO, RECORDS, and STAT screens. The functions of each screen are as follows:
- MAIN: The sleeping video is uploaded, goes through preprocessing, and then ResNet50 classifies the sleep positions.
- INFO: Users can check the pros, cons, and improvements for 10 different sleeping positions.
- RECORDS: Classified images are recorded.
- STAT: A representative image and two charts are provided.
| Screen Name | .java (@java) | .xml (@layout) | Additional files (@java, @drawable) |
|---|---|---|---|
| MAIN | FirstActivity.java | first.xml loading.xml |
FileHelper.java FileHelper2.java border.xml border_red.xml |
| INFO | InfoActivity.java | info.xml | back.jpg back_hurray.jpg back_left.jpg back_right.jpg front.jpg front_hurray.jpg front_left_raised.jpg front_right_raised.jpg left.jpg right.jpg |
| SOURCE | SourceActivity.java | source.xml | |
| RECORDS | RecordActivity.java | records.xml dialog_image.xml |
|
| HISTORY2 (RECORDS) | History2Activity.java | history2.xml dialog_image.xml |
|
| STAT | MainActivity.java | capstonedesign.xml | border.xml border_chart.xml |
| HISTORY (STAT) | HistoryActivity.java | history.xml | border.xml border_chart.xml |
border_info.xml is not used.
When you first run the app, this screen appears. Only the 'INFO' and 'SELECT' buttons are activated.
When you click the 'SELECT' button, you can select a video. After selecting the video in your gallery, the video name is displayed in a black box on the MAIN screen, and the toast message 'μμμ΄ μ λ‘λλμμ΅λλ€.' appears. And then, the 'CLASSIFY' button is activated.
When you click the 'CLASSIFY' button, the images are extracted from the uploaded video in designated frame units. The video path, classification results, and extracted frame units are written and stored in two txt files. After classifying, the 'RECORDS' and 'STAT' buttons are activated.
The txt files are written as shown in the picture above. One is used in the INFO, RECORD, and STAT screens, and the other is used in the HISTORY and HISTORY2 screens. Up to 30 txt files used in HISTORY and HISTORY2 are stored, and more are stored in place of the oldest file.
When you click the 'INFO' button on the MAIN screen before classifying, this screen appears.
When you click the image in the table below, the pros, cons and improvement information of that sleep position are displayed in a dialog window. When you click the 'SOURCE' button, you can see the sources for the images and information.
When you click the 'INFO' button on the MAIN screen after classifying, the classified classes are displayed at the top, and the classe names written in the table are changed to bold.
When you click 'RECORDS' button on the MAIN screen after classifying, you can see the image extracted for each frame and the classification class of the image. If you click any image, the image will be enlarged in the center of the screen.
When you click 'HISTORY2' button on the RECORDS screen, you should choose one txt file. After that, the classification information in the selected txt file is output as if displayed on the RECORDS screen.
When you click 'STAT' button on the MAIN screen after classifying, you can see the most classified class and representative image, a chart of changes in sleep position over time, and a chart of sleep position percentage indicators.
Like the HISTORY2 screen, when you click 'HISTORY' button on the STAT screen, you should choose one txt file. After that, the classification information in the selected txt file is output as if displayed on the STAT screen.