Train model

[LLeiSong]

Train a DL model based on pixel-based labels.

[LLeiSong]

The primary training shown some problems:

• The DL model is sensitive and might be stuck to the noises of labels.
• Due to both the convolution of DL and the defective labels, the predictions are significantly lack of details, even though they have high accuracy.
• The visualization of the predictions doesn't really match the high accuracy.

In order to solve these issues. We tried and will try:

• Try to cooperate other information. OSM is our choice. Using OSM vectors to clean the consensus labels.
• Run RF first as a guess model to get some pre-trained labels. Then select tiles with high certainty to check manually, and give a score to this tile. So finally we will get a sequence of tiles of labels with score tab.
• Then train a co-teaching DL model based on these scored pre-trained LC labels.

[LLeiSong]

Some updates:

• The usage of OSM significantly improves the quality of ensemble labels.
  • We used OSM as a mask to mask out some potential area of bad labels. For instance, roads and buildings with a buffer.
  • We also used some OSM features to fill in the labels. For instance, buildings as new labels of urban, and roads as new labels of bareland. Waterbodies as well.
• RF as a guess model also significantly improve the quality of ensemble labels. The results of RF guess model are good enough as new LC product. However, it is very slow to predict on new tiles. And it can't be transferred as easily as DL.

So, next step we will manually check and grade the guess labels of RF, and then fit them into a co-teaching DL model to train a ready-to-use DL model.

[LLeiSong]

Update Apr 20, 2021:

• Regular training seems work just fine. It is slow without a decent GPU instance though.
• Next step is to implement multiple methods of co-training which includes co-teaching, co-teaching-plus, JoCoR.

[LLeiSong]

Update May 3, 2021:
Hyper-parameter tuning: optimizer, learning rate (includes using scheduler), batch size, and probably transformation methods.

[LLeiSong]

Update May 9, 2021:
Use AdaBound optimizer (https://github.com/Luolc/AdaBound/blob/master/adabound/adabound.py), a learning rate scheduler, 32 batch size, and random flip sync transformation and normalization image transformation to train 200 epochs as a baseline.

Learning rate scheduler:

And training progress:

Evaluation:
'AverageAcc': 0.8917790560347527
'weightedPrec': 0.9230187367229913
'weightedRec': 0.8719084136833852
'weightedF1': 0.8910451918369202
'weightedF2': 0.8769002324638219
'HammingLoss': 0.12809158631661477

Model performs poorly on class urban and barehand based on metrics. Because there are much less labels for these two classes.

Here is a prediction for a whole tile (4096 * 4096):

Based on the results so far, the method is a big success. Next step is to apply multiple methods to use noisy labels.

[LLeiSong]

Update May 16, 2021:
Finish adding all co-teaching related loss functions. Will start to do experiments of how to use them to improve model performance.