Overview

This respository is used as the final project for the course "Deep Learning" on opencampus in the Winter semester 2020/21.

Topic: Image Segmentation
Details: This project managed to finish the task of image segmentation of the Kaggle Challenge, Carvana Image Masking Challenge. The idea came from the interests of knowing how to segment the object from the image. The datasets from the Carvana Image Masking Challenge is based on high quality car photos and the backgrounds of the images usually contain similar colors as cars.
Goals: Work through the challenge and get to understand how image segmentation works, to know what are the state-or-art methods using for image segmentation, and to fine tune the image segmentation method with the knowledge we have learned from the course.
Dream: This project initially planned to finish the Cloud Segmentation challenge on Kaggle, but with the time limitation and busy schedules...

Background Knowledge for Image Segmentation

• Introduction to Image Segmentation
• The Image Segmentation Tutorial on Colab

Possible datasets for image segmentation

1. Some challenges on Kaggle:
   • 38-Cloud: Cloud Segmentation in Satellite Images
   • 95-Cloud: Cloud Segmentation on Satellite Images
   • Ships in Satellite Imagery
   • Understanding Clouds from Satellite Images
   • Carvana Image Masking Challenge
2. The GitHub Repository collecting some satellite imagery datasets:
3. Image Segmentation tutorial with Oxford-IIIT Pet Dataset
4. Open Images 2019 - Instance Segmentation
5. Segmentation evaluation database
6. A Large-scale Dataset for Instance Segmentation in Aerial Images

Link the shared Google drive for dataset we used

• Google Drive: right click the shared folder and click on "Add a shortcut to Drive" to make sure you can easily reach the folder
• Instruction for loading data in Google drive to Google Colab

from google.colab import drive
drive.mount('/content/drive')
%cd /content/drive/MyDrive/DL_project/kaggle/data/

Possible Algorithms

• U-Net
  • Origin paper for U-Net
  • Introduction about U-Net
• FractalNet
  • Background Paper about FractalNet
  • A Repository for FractalNet implementation in Keras
• Others: A repository for Deep Segmentation with several CNNs for semantic segmentation (U-Net, SegNet, ResNet, FractalNet) using Keras

Notes for some possible further applications

• Optical flow
• Using Machine Learning to “Nowcast” Precipitation in High Resolution
• A Neural Weather Model for Eight-Hour Precipitation Forecasting

Notebooks

• Examples to work from: DL_Images.ipynb
• Final Notebook: Image_segmentation.ipynb

Overview of the timeline for this project

14th Dec. - 4th Jan.: project choice, dataset pre-processing, maybe first simple model and objective
4th Jan.: Peer review session, each group present their status to another group
4th Jan. - 25th Jan.: Architecture, fine-tuning, preparation presentation

Before Peer Review on 4th Jan.

Main task: Find an intermediate target. It sounds a bit too hard to reach the target of cloud segmentation in six weeks. So before the peer review, we should choose the dataset and have objectives.

1. Look into different training datasets
2. Try to train them with some exist networks (for example, revising the Image Segmentation tutorial to train the dataset you find)
3. List down your findings:
   • Datasets: What you find? How it works with the networks that you used? Any interesting notebooks you find? What might be the challenging part?
   • Possible networks: What kind of application the networks mostly used for? What are their architecture? Any explanation for them? (And feel free to upload the program you wrote, it would be nice for the other to test)
   • Any useful documents you think it might help our project
4. Have another discussion before 4th Jan.

Before meeting with Luca on 18th Dec. 16:30

• All: Look into the Image Segmentation tutorial with Oxford-IIIT Pet Dataset
• EJ: Check how to link Colab with GitHub repository
  • add file from github: simply click the link, check the buttom "Private Repositories einschließen", and select this repository
  • push the file to github: file > Save a copy in Github
• Sebastian: Meeting tools
• Suman: Look into the dataset from Kaggle and give a brief summary
• Erwin: Work through the dataset from Kaggle and the possible applications

Requirements of the final project

Date: 18th Jan. 2021 Time: 10 minutes for presentation + 5 minutes for a round of questions

Presentation structure:

• Group: who are inside the group
• Project: short description of the project and the motivation behind
• Tools (optional)
• Architecture
• Story (optional): attempts, failures, successes
• Results
• Baselines - how to measure the performance? is it good enough?? compare to??
• Missing (optional) - is there something you missed to improve in the project?
• Future work (optional) - how to improve

• Sharing (optional) on the opencampus gitbook: Code, Data and Presentation