Skip to content

Repo for the ML/DL assignment in the CAS Machine Intelligence 2019 (ZHAW)

Notifications You must be signed in to change notification settings

grssnbchr/cas-mi-mldl-assignment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Land cover classification from airborne imagery

Instructions

  • The ML assignment resides in main.py and the write-up in notebooks/ML\ Assignment.ipynb. Anaconda was used to install the requirements in requirements.txt.
  • The DL assignment is (only) a notebook which resides in notebooks/DL\ Assignment.ipynb. The poster (Scribus file) resides in dl_poster.
  • In order for scripts/notebooks to run, the data folder needs to be created and populated with a deepsat-sat6 folder that contains the training and testing data as extracted from the Kaggle download.

Important links

Resources

Tutorials

Shallow learning

Results

10% of the data, 0.8:0.2 train:validation

No preprocessing, 3136 features

  • RF with 100 trees: 96%

No preprocessing, but NVDI added, 3920 features

  • RF with 100 trees: 97% (training took ca. 75sec)

Only grayscale features, 784

  • RF with 100 trees: 83.9%

Standardization, without NVDI, 3136 features

  • RF with 100 trees: 96%

Standardization, stats extracted, 8 features

  • RF with 100 trees: 98.8%
Percentage correct:  98.79629629629629
              precision    recall  f1-score   support

           0       0.95      0.97      0.96       311
           1       0.99      0.98      0.99      1430
           2       0.99      1.00      0.99      1111
           3       0.98      0.97      0.98      1034
           4       0.94      0.91      0.92       149
           5       1.00      1.00      1.00      2445

   micro avg       0.99      0.99      0.99      6480
   macro avg       0.97      0.97      0.97      6480
weighted avg       0.99      0.99      0.99      6480

Standardization, NVDI, stats extracted, 10 features

  • RF with 100 trees: 99.0% (5 secs training)
  • The same with only 1% of training data: 97.5%
Percentage correct:  98.99691358024691
              precision    recall  f1-score   support

           0       0.97      0.97      0.97       311
           1       0.99      0.99      0.99      1430
           2       0.99      1.00      0.99      1111
           3       0.98      0.98      0.98      1034
           4       0.94      0.95      0.94       149
           5       1.00      1.00      1.00      2445

   micro avg       0.99      0.99      0.99      6480
   macro avg       0.98      0.98      0.98      6480
weighted avg       0.99      0.99      0.99      6480
  • Adding NVDI made the f1-score for the 4th class a bit better by 2 percentage points

Standardization, NDVI, stats extracted, optimized model after randomized grid search

  • Validation accuracy: 99.0%
  • Test set accuracy: 99.0%
  • winning params: {'statsextractor': StatisticsExtractor(), 'standardizer': None, 'rf__n_estimators': 1788, 'rf__min_samples_leaf': 1, 'rf__max_features': 'sqrt', 'rf__max_depth': 50, 'rf__bootstrap': False, 'nvdiadder': AddNVDI()}

Deep learning

Results

All examples without preprocessing (except normalization) and all 4 base channels.

  • Simple CNN with ~700k params, after 30 epochs (40mins training):
test set accuracy according to sklearn.accuracy_score: 0.9734320987654321
              precision    recall  f1-score   support

           0       0.94      0.93      0.93      3714
           1       0.97      0.97      0.97     18367
           2       0.99      0.97      0.98     14185
           3       0.92      0.95      0.94     12596
           4       0.86      0.89      0.87      2070
           5       1.00      1.00      1.00     30068

   micro avg       0.97      0.97      0.97     81000
   macro avg       0.95      0.95      0.95     81000
weighted avg       0.97      0.97      0.97     81000
  • Best model after 2 hours of neural architecture search (NAS):
test set accuracy according to sklearn.accuracy_score: 0.9922592592592593

              precision    recall  f1-score   support

           0       0.99      1.00      0.99      3714
           1       0.99      0.99      0.99     18367
           2       1.00      0.99      0.99     14185
           3       0.97      0.98      0.98     12596
           4       1.00      0.97      0.98      2070
           5       1.00      1.00      1.00     30068

   micro avg       0.99      0.99      0.99     81000
   macro avg       0.99      0.99      0.99     81000
weighted avg       0.99      0.99      0.99     81000

About

Repo for the ML/DL assignment in the CAS Machine Intelligence 2019 (ZHAW)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published