September Iteration Plan

[QuanluZhang]

This is the draft plan for the upcoming iteration in September, it's a 4 weeks iteration.

Release Plan for v1.9

• Release manager: @QuanluZhang
• Feature freeze date: 9/22 9/23
• Code freeze and demo date: 9/29 10/9
• WebUI specific: 10/12
• Retrospective date: 10/13
• Release and next iteration plan date: 10/19 10/20 10/21

---

Let's try to adopt the legend of annotations:

Mark	Description
🏃‍♂️	work in progress
👪	work under review
✋	blocked task
💪	stretch goal for this iteration
🔴	missing issue / pull request reference
🔵	more investigation and survey required
⚫	under discussion within the team
⬛	a large work item, larger than one iteration

NAS

• Support more tuning algorithms for classic NAS (regularized evolution) @tabVersion
  • regularized evolution, 9/22, pr [tuner] Regularized Evolution #2802 , reviewer @ultmaster @chicm-ms @QuanluZhang
• NAS bench 201 in search space zoo @tabVersion pr Search Space Zoo nas bench 201 #2766 reviewer: @ultmaster @QuanluZhang

Model Compression

• Model compression doc refactor @chicm-ms (survey) 9/22, pr Refactoring model compression doc #2919 , reviewer @QuanluZhang @colorjam

• Quantization doc refactor, algorithm and framework review, support post-train quantization speedup @linbinskn
  1. review supported quantize algorithms, 9/7: Dorefa fix, reproduction of supported quantization algorithms

  • discussion of current quantization framework 9/7, conclusion: no need to refactor (done).
  • update quantization doc 9/14
  • support bias in quantization algorithms, pr Add bias quantization in QAT and refactor the code of weight quantization #2914 , reviewer: @chicm-ms @QuanluZhang

• AMC improvements @chicm-ms

  • support resnet: 9/9 9/11 9/16 9/17, pr AMC supports resnet #2876 , reviewer: @QuanluZhang @zheng-ningxin
    Remove the restriction of kernel size must be 1x1, support larger kernel size.
    Integrate NNI's automatic dependency extraction instead of hard coded dependency.
    Use NNI's speedup framework in AMC to replace the existing one.
  • use paper reproduce parameters as default parameters for the example, pr Speedup supports channel pruning #2906, reviewer: @QuanluZhang @zheng-ningxin

• flops counter refactor, pr ready Add custom op support of counter #2795 @colorjam reviewer: @chicm-ms @liuzhe-lz

• Update TF model compression @liuzhe-lz

  • support sequential 9/9 , pr Compression support for tf.keras.Sequential #2887 , reviewer: @chicm-ms @QuanluZhang

• constraint-aware pruner @zheng-ningxin pr Constraint-aware one-shot pruners #2657 reviewer: @chicm-ms @QuanluZhang

WebUI & NASUI

• Support visualization and management of multiple experiments on WebUI @yangou1988 @Lijiaoa @sharonxingyufan
  • design, 2 days, done
  • improve UI @Lijiaoa 9/18
    • new overview page 9/22, pr [webui] refactor overview page #2924 reviewer: @ultmaster @colorjam @liuzhe-lz
  • upgrade node and yarn 9/16 @ultmaster pr Upgrade node and yarn #2894
  • upgrade Fabric UI @Lijiaoa 9/7 pr ready Update webui react version and use fluent ui #2873 , reviewer: @liuzhe-lz @ultmaster
  • ESlint @Lijiaoa pr [webui] format code by eslint and prettier #2744 reviewer: @ultmaster @liuzhe-lz
• improvement of tensorboard support @yangou1988
  • WebUI design @yangou1988 @sharonxingyufan 9/14
• Add/Remove columns in hyper-parameter chart and trial details table and Refactor trial details table (click event) @ultmaster 9/18 9/20 9/23 10/6 pr Chevron icon before table row and TableList refactoring #2900
  • choose trial will be broken when click table pagination @ultmaster @Lijiaoa 9/20 9/23 10/6
• JSON format utility beautify on WebUI @liuzhe-lz 9/3 PR Pretty JSON formatter #2863 reviewer: @Lijiaoa @ultmaster

nnictl

• support adding description to experiment to make users easily identify a certain experiment @SparkSnail 9/9 9/18 9/23 9/25 9/27 (design)
  • need followup discussion and work items
• Nnictl auto-complete @JunweiSUN 9/7 pr ready, Update bash-completion #2857 reviewer: @liuzhe-lz @SparkSnail
• reconsider the name of nnicli, update doc, NNI rest python client @scarlett2018

training service

• Configure python environment in remote mode @J-shang 9/7 9/8, pr Add preCommand option to support configuring experimental environment by user #2875 , reviewer: @SparkSnail @chicm-ms , related issue GPU unable to be used in remote mode #2366
• adapt remote training service to the reuse framework @SparkSnail 9/22, Support remote training service use reuse mode #2923 , reviewer: @chicm-ms @squirrelsc @tabVersion
• aml mode under Windows @J-shang pr modify the pythonPath of PythonShell to support aml mode under Windows #2882 , reviewer: @SparkSnail @liuzhe-lz

UT & IT

• Add IT cases: @J-shang
  • Import/export 9/9, pr fix experiment import bug and add it cases: experiment import #2878 , reviewer: @chicm-ms @ultmaster
  • User installed builtin tuner, pr ready, reviewer: @chicm-ms @liuzhe-lz
• Nnictl UT @SparkSnail 9/18, pr Add nnictl ut #2912 , reviewer: @chicm-ms @ultmaster

Documentation

• table rendering issue in Chinese doc @tabVersion @JunweiSUN
• fix readthedocs warning issue @tabVersion @JunweiSUN
• fix readthedocs build error @ultmaster 9/18
• doc solution for restful apis, use sphinx-js @tabVersion 9/22 discussion done, 9/30
  • upgrade sphinx @tabVersion 9/30

Others

• fix Dependabot alerts @chicm-ms 9/27

bug fix

• Bug fix of naïve evolution tuner @colorjam , 9/8, discussion done, pr Fix evolution tuner RuntimeError and IndexError #2695 , reviewer: @QuanluZhang @tabVersion @chicm-ms
• "WARNING (nni.protocol) IPC pipeline not exists, maybe you are importing tuner/assessor from trial code?" @liuzhe-lz 1 day PR: Silence IPC environment warning in nnictl config validator #2864 , reviewer: @chicm-ms @SparkSnail
• experiment save/load: search space issue @SparkSnail 9/11 pr Support saving search space in experiment load&save command #2886 reviewer: @ultmaster @chicm-ms
• experiment import bug @J-shang 9/9, pr, reviewer: @QuanluZhang @ultmaster
• annotation in remote mode, python 3.8 ast update issue, @J-shang pr support annotation in python 3.8 #2881 @liuzhe-lz @QuanluZhang

To be discussed

• review nnictl metadata management; experiment resume and save/load @SparkSnail @QuanluZhang , before 9/11 9/16, 9/23 review meeting

================

Bug Bash

NAS

• regularized evolution, tester: @ultmaster
• NAS bench 201 in search space zoo, tester: @ultmaster

Model Compression

• Model compression doc refactor, tester: @QuanluZhang

  • update quantization doc 9/14

• support bias in quantization algorithms, tester: @chicm-ms

• AMC improvements tester: @colorjam

  • support resnet: pr AMC supports resnet #2876 ,
    Remove the restriction of kernel size must be 1x1, support larger kernel size.
    Integrate NNI's automatic dependency extraction instead of hard coded dependency.
    Use NNI's speedup framework in AMC to replace the existing one.
  • use paper reproduce parameters as default parameters for the example, pr Speedup supports channel pruning #2906,

• flops counter refactor, tester: @chicm-ms

• support sequential tester: @chicm-ms

• constraint-aware pruner, tester: @chicm-ms

WebUI & NASUI

• new overview page, tester: @QuanluZhang
• Add/Remove columns in hyper-parameter chart and trial details table and Refactor trial details table (click event) tester: @SparkSnail
• JSON format utility beautify on WebUI tester: @Lijiaoa

nnictl

• Nnictl auto-complete, tester: @liuzhe-lz
• reconsider the name of nnicli, update doc, NNI rest python client @scarlett2018

training service

• Configure python environment in remote mode, tester: @SparkSnail
• adapt remote training service to the reuse framework, tester: @J-shang
• aml mode under Windows, tester: @QuanluZhang

Documentation

• table rendering issue in Chinese doc @tabVersion @JunweiSUN
• fix readthedocs warning issue @tabVersion @JunweiSUN
• fix readthedocs build error, tester: @tabVersion
• doc solution for restful apis, use sphinx-js @tabVersion 9/22 discussion done, 9/30
  • upgrade sphinx @tabVersion 9/30

🐛 bug fix

• Bug fix of naïve evolution tuner, tester: @J-shang
• "WARNING (nni.protocol) IPC pipeline not exists, maybe you are importing tuner/assessor from trial code?", tester: @J-shang
• experiment save/load: search space issue, tester: @ultmaster
• experiment import bug, tester: @QuanluZhang
• annotation in remote mode, python 3.8 ast update issue, tester: @liuzhe-lz