Capstone Team C22-PS135 [RendangBakar Team]

Bangkit Capstone Project 2022: HealthLens (Deteksi Berbagai Penyakit Wajah)

~ Mind Your Skin Health ~

· Report Bug

Table of Contents

1. About The Project
   • Built With
   • Library
2. Documentation
   • Machine Learning
   • Cloud Computing
   • Mobile Development
     • Prerequisites
     • Installation
     • How to use
3. Contact us

About The Project

Nowadays, people are becoming aware about their facial health. Especially about their facial problems, specifically, like their skin doesn't look healthy, have acne, and problem skin-related.There are many kinds of skincare products with many variants like products, ingredients, and usefulness. The problems are people find it hard to identify which one skincare product is right for them because there are so many problems related to facial skin. The people aware of the skin do not look healthy, have acne, or problem skin.

Thus, there are chances that people don't use the skincare product wisely and suitable for them and only just make their facial problem more severe for themselves. Chances are high because sometimes people use skincare without consulting the professional, before. Mainly because of the high cost for them and there is a lack of professional facial doctors who could resolve the problem in their regions or residence. Briefly to conclude, we need some automation tools that could meet human professional-like intelligence for resolving the problems without being needed to consult a professional facial-skin doctor.

Built With

• Android Studio Chipmunk (2021.2.1 Patch 1)
• Kotlin Programming Language
• Google Cloud Platform Services
• Google Colaboratory
• Python Programming Language (Version 3)

Library

• Pandas
• Numpy
• Tensorflow
• Keras
• Flask
• Zipfile
• Retrofit
• Glide
• Canhub Image Cropper

Documentation

Machine Learning

Making the model classification

The models used for this classification are multi-class classification which's contains four classes for each part of classification. The classifier was built using Artificial Neural Network which built in TensorFlow API and use of InceptionV3's transfer learning for enhance the performance. The classifier we build made of approximately four hundreds (400) of datasest with each of classes contains 100 images (90:10 split for training and validation test). Datasets are collected manually by using several search engine website, free-popular website images (unsplash, freepik, etc.), and some manually images generated by the team. Before training, the dataset going to the preprocessing cycle for cleaning the datasets.

The Classifier approximately have accuracy up to 75% for skin disease and 80-82% for skin type. Still not good but still decent. All the notebooks code, datasets, and model weight for these purposes are provided in this repository.

Cloud Computing

1. Python 3.7
2. Flask
3. Numpy, Keras. TensorFlow, Pandas, (requirement.txt)

How to Run

1. use virtualenv and run flask with python
2. and goto "ip"/upload

Endpoint

http://104.197.16.252/

Upload

• URL
  • /upload
• Method
  • POST
• Headers
  • Content-Type : multipart/form-data
• Request Body
  • picture_path as file
  • kind_model as string
• Response

{
    "error": false,
    "message": "success",
    "id": "1654407149"
}

Result

• URL
  • /result
• Method
  • GET
• Parameters
  • id as string
• Response

{
  "error": false, 
  "id": "1654407149", 
  "message": "success", 
  "productList": [
    {
      "linkProduct": "https://www.tokopedia.com/synergypusat/elemence-vera-gel", 
      "name": "Elemence Vera Gel", 
      "photo": "/home/a7009f0996/Gambar/24_Wrinkles/1.png"
    }
  ], 
  "rekomendationList": [
    "Using aloe vera gel", 
    "use vitamin C in the form of L-ascorbic acid", 
    "Avoid sun exposure", 
    "Use the right skin care products", 
    "Use a serum that contains retinol", 
    "Drinks that contain collagen"
  ], 
  "resultDetection": "Wrinkles"
}

Mobile Development

Prerequisites

1. Android 5.0 Lollipop (SDK 21) or above.
2. Internet Connection.
3. Good condition Front Camera.

Installation

1. Download the APK Files here.
2. Install APK FIles (Allow permission install from Unknown Source).

How to use

1. Open App.
2. Press the "Get Started" botton on Landing Page.
3. Select the desired Detection Type (Skin Disease or Skin Type).
4. Press the "Take a selfie" button to take a photo from the front camera, or press the "Upload from gallery" button to select a photo from the gallery.
5. If you choose "Take a selfie", you must give permission to use the camera on this app. Then take a photo by pressing the shutter button.
6. If you choose "Upload from Gallery", select the photo you want to detect, then crop the image and point it directly at your face.
7. if you want to re-take the photo, you can press the "Try Again" button. If not, press the "Procces" button to start the detection.
8. Please wait a moment, let HealthLens detect your facial skin and provide you with accurate results!
9. Now, The prediction about your skin type or skin problem has been out! You can also see recommendations for what to do to take care of your face and product recommendations based on your facial condition!

Contact us

• Miftahul Ardli (M2011G1151) - miftahul18002@mail.unpad.ac.id
• Ni Luh Melika Candra Widyani Mas (M2232F2123) - melika.candra@gmail.com
• Alvin Steven (C2007F0655) - alvin.pin108@gmail.com
• Muhammad Alwi Abdul Aziz (C7007F0656) - 112201906316@mhs.dinus.ac.id
• Tegar Naufal Hanip (A7009F0996) - tegarnaufalh@gmail.com
• Vinna Mariska (A2009f0997) - vinna.mariska@gmail.com