Iteration Plan for Jan-Feb 2020

[scarlett2018]

Release Manager: @QuanluZhang
Feature freeze date: 2020/1/23
Code freeze date: 2/10 4PM
this will be a 5-week sprint , mid sprint break: 1/24-1/30 Chinese Spring Festival
End Game Issue 2/12-2/17: is tracking in Jan-Feb 2020 End Game Plan #2025
Target release date: 2/18/2020

---

Open discussion / retrospective from previous release

• Improve endgame process, i.e. auto IT for release branch. Follow up: add this to the release manual @chicm-ms @leckie-chn
• Stay with feature freeze deadline, to avoid big PR merge's risk by the end of release
  • problem: NAS algo might cause too much time to get the results (to prove the algo quality); resolutions: we might think of silent/experiment release for core functions.

---

Planned Item

• Finalize the integration with PT @SparkSnail @ultmaster @scarlett2018
  • support merge user PT config file with NNI config file @SparkSnail (PR ready for review in internal repo @QuanluZhang @chicm-ms )
  • build pypi for internal version @SparkSnail (pending for the above PR)
• Proxyless NAS left work - reproduce and report the results (2/3 finish check) @QuanluZhang support proxylessnas with NNI NAS APIs #1863 [need review Chi]

TensorFlow 2.0 support @liuzhe-lz (PR eta. 2/6 9)

• [ ] tensorflow 2.0 (tf.keras) support for NNI NAS interface (Using NNI NAS interface with keras/tf.keras #1737 ) (1.8 pr) @liuzhe-lz

• tensorflow 2.0 support for static dynamic graph model compression (depend on Using NNI NAS interface with keras/tf.keras #1737 ) @chicm-ms

• fabric framework adoption @lvybriage (ETA 2/7)

• cancel trials in UNKNOWN state (Feature request: UNKNOWN state - ability to cancel trial #1735 , pr submitted, ) @chicm-ms @lvybriage

• cancel trials in UNKNOWN state WebUI (pr submitted ) @lvybriage

• BUG - correctly support NaN in metric data, JSON compliant (ValueError: Out of range float values are not JSON compliant #1589 , 1.14 pr, ready for review)@chicm-ms

• add a parameter to cmd to support nni command as top cmd (1.13 pr, ready for review @chicm-ms ) @SparkSnail

• add pipeline for frameworkcontroller (1.17 PR1971 @chicm-ms @liuzhe-lz ) @SparkSnail @chicm-ms

• support complete PAI configuration file (1.17 pr @chicm-ms) @SparkSnail

• update pai doc @SparkSnail

Model compression

• Support DataParallel (How to use multiple gpus for pruner #1915 , committed to Dev Branch) @Cjkkkk
  • update 4 quantizers, and ActivationAPoZ/Mean, Level 3 pruners and @Cjkkkk [PR submitted, need a discussion with @QuanluZhang and @chicm-ms at 4pm 2/7 to discuss defer or not; need to merge dev to master (2/9);]
  • update L1/2Filter, FPGM, AGP 4 pruners @chicm-ms
  • update Lottery Ticket 1 pruner @QuanluZhang
  • update UT @chicm-ms
• Model transform for real speed up, an alpha version (flops and parameters question about slim pruner #1846 / 为什么模型压缩后的占用空间没有减小？ #1927 , 1.14 design review, 1.22 draft pr) @QuanluZhang

Neural architecture search (NAS)

• C-DARTS @ultmaster (1.17 pr by Hongyuan, ready for review)
• tests for supported NAS algorithms (Can nni offer some test metric for the example using one-shot/dart/enas model? #1598 , @liuzhe-lz @QuanluZhang PR review) @ultmaster
• tests for data parallel NAS algorithms (Can nni offer some test metric for the example using one-shot/dart/enas model? #1598, @liuzhe-lz @QuanluZhang PR review) @ultmaster @chicm-ms
• prepare GPU vm for pipeline (1.14) @chicm-ms
• fix security issue in win vm @chicm-ms

Documentation

• add readthedoc to build process (Friday pr, ready for review @chicm-ms ) @leckie-chn
• improvement of document structure (ready for review) @QuanluZhang @leckie-chn @scarlett2018 (Refactor the outline of NNI document in readthedocs #1827 To be reviewed)
• documentation for NAS (1.20 pr) @ultmaster

------------------Deferred------

• tensorflow 2.0 support for static graph model compression (2d) @liuzhe-lz
• tensorflow 2.0 support for static graph NAS (1.13 1.14 draft pr 1.17 pr) @liuzhe-lz
• tensorflow 2.0 test and example (code 2d) @liuzhe-lz
• TF2 - upgrade to the new cuda version for our maintained docker image (Support multiple CUDA version in docker images #1748 , 1 day, first test internally) @liuzhe-lz
• Integration with DLWS //@QuanluZhang is following up with Qixiang @Gerhut , need more time.
• bug - clone fail @liuzhe-lz “Unknown error" for clone trial with MNIST example #1985
• AutoNLP @QuanluZhang
  • search algo @ultmaster
  • retrain //yaming
• prepare GPU vm for pipeline (1.14), migrate remote docker to the GPU vm @SparkSnail
• write doc for IPC interface (Write documentation for IPC interface #1741 ) @liuzhe-lz
• customized homepage for sphinx doc build (before code freeze) @leckie-chn @lvybriage
• webui supports NaN/Infinite
• intermediate graph only draw numeric numbers, and fix state (top right button) issue
• clean issues assigned to each member
• IT/UT: resume experiment, trial uses dict metric, experiment view, experiment foreground
• replace pai IT with new ones
• NAS visualization
• speedup example refactor
• refactor the logic of applying mask file
• compression framework refactor
• model compression examples refactor