Give Ocean A Piece Of Your Mind

🇹🇼繁體中文 | 🌏English | ❓issues | 📝Paper

Overview🍩

This system is dedicated to the sustainable development of the ocean, combining underwater image colorization and restoration, object detection, and various neural network models to assist people in more effectively researching and understanding the ocean.

News📢

[2023/12/15] The paper is officially online. For details, please check IEEE Xplore.

Work in progress📋

• Modifying WaterNet to the U-Net architecture and observing its performance. For details, please check the dev branch.
• Transfer the project to the new research team.

Paper📝

Yi Lin; Chung-Wei Hung; Yu-Jie Wang; Chih-Chia Liao; Yu-Shiuan Tsai, "Enhancing Underwater Images: Automatic Colorization using Deep Learning and Image Enhancement Techniques," 2023 IEEE International Conference on Marine Artificial Intelligence and Law (ICMAIL) IEEE Xplore

Advantages

• Multimedia Support: Capable of loading images, videos, and web camera feeds.
• Scene Variety: Provides weights for underwater and land scenes.
• Ocean Sustainability Technology: Assists in the observation and research of the ocean.

Key Features

1. Underwater Image Restoration: WaterNet, using weights trained on the provided dataset by the author.

2. Automatic Colorization: neural-colorization, original weights provided by the author, and weights trained by us.

3. Load Images, Videos, and Web Camera Feeds: Supports multimedia usage.

4. Image Slider Comparison: Provides a more convenient and intuitive way to compare images.

5. Object Recognition: YOLO v8, official original weights and weights trained for fish and colorblock recognition.

6. Colorblock Capture and Analysis: Manually captures colorblocks to evaluate colorblock data in images.

Installation

• To run this project locally, follow these steps:

1. Clone the repository of this project by executing the following command:

git clone https://github.com/NTOU-Arrays-Start-at-One/Give-ocean-a-piece-of-your-mind.git

2. Install the required packages by running the following command:

pip install -r requirements.txt

This will install all the necessary packages for this project.

Note: This project uses python version 3.8.10. If your python version is too different, there may be problems with the packages.

Usage

• Run main.py:

python3 main.py

Notes📔

• Development environment: Ubuntu 20.04.6 LTS
• Windows users need to execute python main.py and replace all instances of python3 in the code with python.

Example:

subprocess.call([
                "python", colorization_path,
                "-i", source_path,
                "-m", weights_path,
                "-o", output_path,
                "--gpu", "-1",
            ]) 

Research Methods

We employ the WaterNet water color restoration model and neural-colorization automatic colorization model for underwater color restoration and black-and-white image colorization. Subsequently, we employ numerical methods to evaluate the image restoration results. Furthermore, we utilize YOLO v8 for object recognition.

1. WaterNet Water Color Restoration Model WaterNet is a model based on convolutional neural networks designed to mitigate the impact of underwater light scattering.

2. Neural-Colorization Automatic Colorization Model Due to severe red light loss underwater, the automatic colorization model has the capability to generate any color in grayscale images. As a result, we apply it to underwater color restoration.

3. Color Analysis In order to compare results before and after image restoration, we incorporate colorboard photos into the shooting scenes to analyze and assess the effectiveness of restoration. We employ k-means and CIE-2000 for color block analysis.

4. Object Recognition Through YOLO v8 object detection, we can identify various objects, fish species, and colorboards.

Other

Potential Issues

• colormath Version Problem: May result in color analysis not functioning correctly.
  • Issue #27: Colorboard Analysis on the First Page of Windows Cannot Be Analyzed Correctly
• NumPy Version Problem:
  • Issue #26: NumPy Runtime Issue Encountered When Capturing Manually
• Window Size Concern: Computers with resolutions below 1080p may have usability issues.
  • Issue #17: Window Size Concerns
• Windows Usage Problems:
  • Issue #6: Errors Running waterNet and Colorization on Windows

Past Versions

• waternet_fasterRCNN: Automatic color board capture and analysis.
• Perspective-control-and-Color-testing: Automatic color board calibration and analysis.

Citations

If you find our work helpful in your research, please consider citing this paper. You can use the following BibTeX entry:

@INPROCEEDINGS{10347502,
  author={Lin, Yi and Hung, Chung-Wei and Wang, Yu-Jie and Liao, Chih-Chia and Tsai, Yu-Shiuan},
  booktitle={2023 IEEE International Conference on Marine Artificial Intelligence and Law (ICMAIL)}, 
  title={Enhancing Underwater Images: Automatic Colorization using Deep Learning and Image Enhancement Techniques}, 
  year={2023},
  pages={48-53},
  doi={10.1109/ICMAIL59311.2023.10347502}}

Acknowledgments

The study is primarily built upon the secondary development of the following open-source projects. We express our gratitude to the associated projects and research developers:

• WaterNet
• neural-colorization
• YOLO v8

Research supported by Ministry of Science and Technology with the grant number MOST-110-2634-F-019-001 - and National Science and Technology Council with the grant number NSTC 111-2634-F-019-001 -.

Furthermore, we appreciate the computational resources and guidance provided by the NTOU CSE Big Data and Deep Learning Lab, as well as the training data provided by the NTOU EE Smart Living Technology Lab.