Fix a few issues in EfficientNet introduced by API change and rephras…

…e documentation (#1926)

* refine efficientnet

* pai -> openpai

* fix interval

* update batch size

* move readme to trial examples

* update to docs

README.md

@@ -88,6 +88,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
            <li><a href="docs/en_US/TrialExample/GbdtExample.md">Auto-gbdt</a></li>
            <li><a href="docs/en_US/TrialExample/Cifar10Examples.md">Cifar10-pytorch</li></a>
            <li><a href="docs/en_US/TrialExample/SklearnExamples.md">Scikit-learn</a></li>
+           <li><a href="docs/en_US/TrialExample/EfficientNet.md">EfficientNet</a></li>
               <a href="docs/en_US/SupportedFramework_Library.md">More...</a><br/>
           </ul>
         </ul>

docs/en_US/TrialExample/EfficientNet.md

@@ -0,0 +1,21 @@
+# EfficientNet
+
+[EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
+
+Use Grid search to find the best combination of alpha, beta and gamma for EfficientNet-B1, as discussed in Section 3.3 in paper. Search space, tuner, configuration examples are provided here.
+
+## Instructions
+
+[Example code](https://github.com/microsoft/nni/tree/master/examples/trials/efficientnet)
+
+1. Set your working directory here in the example code directory.
+2. Run `git clone https://github.com/ultmaster/EfficientNet-PyTorch` to clone this modified version of [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch). The modifications were done to adhere to the original [Tensorflow version](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) as close as possible (including EMA, label smoothing and etc.); also added are the part which gets parameters from tuner and reports intermediate/final results. Clone it into `EfficientNet-PyTorch`; the files like `main.py`, `train_imagenet.sh` will appear inside, as specified in the configuration files.
+3. Run `nnictl create --config config_local.yml` (use `config_pai.yml` for OpenPAI) to find the best EfficientNet-B1. Adjust the training service (PAI/local/remote), batch size in the config files according to the environment.
+
+For training on ImageNet, read `EfficientNet-PyTorch/train_imagenet.sh`. Download ImageNet beforehand and extract it adhering to [PyTorch format](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) and then replace `/mnt/data/imagenet` in with the location of the ImageNet storage. This file should also be a good example to follow for mounting ImageNet into the container on OpenPAI.
+
+## Results
+
+The follow image is a screenshot, demonstrating the relationship between acc@1 and alpha, beta, gamma.
+
+![](../../img/efficientnet_search_result.png)

docs/en_US/examples.rst

@@ -12,3 +12,4 @@ Examples
     GBDT<./TrialExample/GbdtExample>
     RocksDB <./TrialExample/RocksdbExamples>
     KDExample <./TrialExample/KDExample>
+    EfficientNet <./TrialExample/EfficientNet>

examples/trials/efficientnet/README.md

@@ -1,19 +1 @@
-# EfficientNet
-
-[EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://arxiv.org/abs/1905.11946)
-
-Provided here are: Search space and tuners for finding the best tuple (alpha, beta, gamma) for EfficientNet-B1 with grid search, as discussed in Section 3.3 in [paper](https://arxiv.org/abs/1905.11946).
-
-## Instructions
-
-1. Set your working directory here in this directory.
-2. Run `git clone https://github.com/ultmaster/EfficientNet-PyTorch` to clone this modified version of [EfficientNet-PyTorch](https://github.com/lukemelas/EfficientNet-PyTorch). The modifications were done to adhere to the original [Tensorflow version](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) as close as possible (including EMA, label smoothing and etc.); also added are the part which gets parameters from tuner and reports intermediate/final results. Clone it into `EfficientNet-PyTorch`; the files like `main.py`, `train_imagenet.sh` will appear inside, as specified in the configuration files.
-3. Run `nnictl create --config config_net.yml` to find the best EfficientNet-B1. Adjust the training service (PAI/local/remote), batch size in the config files according to the environment.
-
-For training on ImageNet, read `EfficientNet-PyTorch/train_imagenet.sh`. Download ImageNet beforehand and extract it adhering to [PyTorch format](https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet) and then replace `/mnt/data/imagenet` in with the location of the ImageNet storage. This file should also be a good example to follow for mounting ImageNet into the container on OpenPAI.
-
-## Results
-
-The follow image is a screenshot, demonstrating the relationship between acc@1 and alpha, beta, gamma.
-
-![](assets/search_result.png)
+[Documentation](https://nni.readthedocs.io/en/latest/TrialExample/EfficientNet.html)

examples/trials/efficientnet/config_local.yml

@@ -0,0 +1,18 @@
+authorName: unknown
+experimentName: example_efficient_net
+trialConcurrency: 4
+maxExecDuration: 99999d
+maxTrialNum: 100
+trainingServicePlatform: local
+searchSpacePath: search_net.json
+useAnnotation: false
+tuner:
+  codeDir: .
+  classFileName: tuner.py
+  className: FixedProductTuner
+  classArgs:
+    product: 2
+trial:
+  codeDir: EfficientNet-PyTorch
+  command: python main.py /data/imagenet -j 12 -a efficientnet --batch-size 48 --lr 0.048 --wd 1e-5 --epochs 5 --request-from-nni
+  gpuNum: 1

examples/trials/efficientnet/search_net.json

@@ -1,14 +1,14 @@
 {
   "alpha": {
     "_type": "quniform",
-    "_value": [1.0, 2.0, 0.1]
+    "_value": [1.0, 2.0, 0.05]
   },
   "beta": {
     "_type": "quniform",
-    "_value": [1.0, 1.5, 0.1]
+    "_value": [1.0, 1.5, 0.05]
   },
   "gamma": {
     "_type": "quniform",
-    "_value": [1.0, 1.5, 0.1]
+    "_value": [1.0, 1.5, 0.05]
   }
 }

examples/trials/efficientnet/tuner.py

@@ -14,11 +14,11 @@ def __init__(self, product):
         super().__init__()
         self.product = product
 
-    def expand_parameters(self, para):
+    def _expand_parameters(self, para):
         """
         Filter out all qualified parameters
         """
-        para = super().expand_parameters(para)
+        para = super()._expand_parameters(para)
         if all([key in para[0] for key in ["alpha", "beta", "gamma"]]):  # if this is an interested set
             ret_para = []
             for p in para: