[WIP until the conversion issue is resolved] Add EfficientNet

[sdll]

This PR adds the implementation of EfficientNet B0-5 inference pipeline, with the support for quantized weights.

Roadmap

• Translate predicted logits to top K classes

• Test for memory leaks and validate inference

• Add the demo

• Improve the docs

• Make a script converting original checkpoints to TF.js JSON
  UPDATE 18/06: I have made a PR with the script to the most popular Keras implementation.
  UPDATE 25/06: The conversion tf.keras -> SavedModel -> TF.js Graph Model works well.
  UPDATE 08/07: The conversion to Keras in the first step turned out to be unnecessary, so I simplified the script, but could not handle a pesky bug with the converter (exporting to Keras first did not help as well, so the optimisation was not the source of the issue).

• Integrate the TF Checkpoint to Keras converter script into the workflow for end-to-end porting of weights to TF.js JSON
  UPDATE 18/06: To make the model work, we need to implement two custom layers and an initializer.

• Fix the conversion issues

cc: @manrajgrover

---

This change is 

[manrajgrover]

Hi @sdll,

Awesome work here! 💯 I've partially reviewed the PR. There seems to be an issue in conversion script.

Kindly look into the same and I'll review the rest of the PR once fixed.

[manrajgrover]

We should source the checkpoint from an official tfjs bucket and remove dependence on the forked repository

[manrajgrover]

I think we should move these classes to a separate JSON file and import it here.

[sdll]

This seems to be more natural, since otherwise we have to cast class labels to integers, losing the information stored in the data itself.

[manrajgrover]

This script throws the following error

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~ ............................. ~
~ Converting efficientnet-b0... ~
~ ............................. ~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
  File "/home/manrajsingh/github/tfjs-models/efficientnet/dist/efficientnet-source/scripts/load_efficientnet.py", line 156, in <module>
    import eval_ckpt_main
ModuleNotFoundError: No module named 'eval_ckpt_main'

Kindly look into the same

[sdll]

Recent updates to tensorflowjs_converter have introduced a bug which I could not fix for hours. I have tried to roll back (and forth!), but to no avail. The issue is explained here.

[tafsiri]

@sdll A few quick questions. Apart from the conversion issues (I'll try to reproduce that locally when I get a chance), I did have some issues running the build (incorrect reference to preamble) and running the demos (no keys in efficientnets variable thus non of the buttons work). I did hack it in, but didn't get a meaningful prediction. Are those parts still work in progress?

At a higher level I was wondering how big the weights are (in MB)? And what kind of inference times you are getting?

[sdll]

Sorry, @tafsiri, for the confusion, I glossed over changes I had to do following those based on @manrajgrover’s suggestions. This is fixed now, I have successfully published the module locally and tested each of the variants. Here is the screenshot:

Let me know if there are any issues other than those outlined here.

Models quantized to 2 bytes weigh from 10.2 MB (b0) to 58.2 MB (b5).

With regards to performance, b0 is 4.8 s on the cold first run and 0.52 s on the second. The b5 variant is slower, taking 7.9 s on the first run, and 3.2 s on the second.

I will refactor the code to follow the factory pattern of loading the model, taking inspiration from your review of DeepLab.

Thanks for your time and help!

[sdll]

Reviewable status: 0 of 1 approvals obtained (waiting on @manrajgrover and @sdll)

---

efficientnet/README.md, line 21 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

const efficientnet = new EfficientNet(model, isQuantized);

Done.

---

efficientnet/README.md, line 28 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

This should be a relative URL to the source file.

Done.

---

efficientnet/README.md, line 48 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

We should also pass topK here as an example.

Done.

---

efficientnet/README.md, line 58 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

EfficientNets rely on AutoML and compound scaling to achieve superior performance without compromising resource efficiency. The [AutoML Mobile framework](https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html) has helped develop a mobile-size baseline network, **EfficientNet-B0**, which is then improved by the compound scaling method to obtain EfficientNet-B1 to B7.

Done.

---

efficientnet/README.md, line 60 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

Let's format this html code

Done.

---

efficientnet/README.md, line 82 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

./scripts/convert_efficientnet.sh --target_dir=dist --use_venv=t

We will need to fix this after the issue with the converter is resolved (might need to switch to the docker-based pipeline to avoid issues on OS X, for example).

---

efficientnet/README.md, line 88 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

./scripts/convert_efficientnet.sh -h

Done.

---

efficientnet/demo/src/index.js, line 27 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

Let's add keys to Object dynamically during initialization.

Done.

---

efficientnet/scripts/convert_efficientnet.sh, line 145 at r1 (raw file):

Previously, manrajgrover (Manraj Singh) wrote…

We should source the checkpoint from an official tfjs bucket and remove dependence on the forked repository

Done.

[sdll]

Reviewable status: 0 of 1 approvals obtained (waiting on @manrajgrover and @sdll)

---

efficientnet/src/types.ts, line 25 at r5 (raw file):

    |ImageData|HTMLImageElement|HTMLCanvasElement|HTMLVideoElement|tf.Tensor3D;

// #TODO: Fix this after the issues with the converter are resolved

After the conversion issue is resolved, I will add support for all kinds of quantization.

[tafsiri]

@sdll just fyi i get the same issue with calls to hasLoaded() throwing an error that i do in the deeplab demo.

Thanks for the model info

[sdll]

@tafsiri, a fresh install did not show any issues:

# cd tfjs-models/efficientnet
# git pull
cd demo
rm -rf node_modules/ .yalc/ .cache/
cd ..
yarn publish-local
cd demo
yarn
yarn link-local
yarn watch

One thing that I did notice though was that labels were off in Chrome due to the differences in CSS rendering:

This is fixed now: