[WIP until the conversion issue is resolved] Add EfficientNet #238
Conversation
* upstream/master: Update posenet lock file. (tensorflow#244) PoseNet - Split pose estimation methods into estimateSinglePose and estimateMultiplePoses, and move inputResolution specification from loading to inference time. (tensorflow#242) update peer deps for toxicity model and increase the minor version (tensorflow#243) update models peerDependencies to be individual npm instead of union (tensorflow#237) [speech-commands] Bump version to 0.3.9 (tensorflow#240) [speech-commands] Enable creating recognizer from artifacts (tensorflow#226) Update README.md (tensorflow#239) export Mobilenet as interface (tensorflow#236) skip unneeded operation for posenet single-pose allow user to provide a custom url for the mobilenet model (tensorflow#235)
aptlin
changed the title
* upstream/master: Optimize the bundle of our demos (tensorflow#247) Fix the broken gif in PoseNet. (tensorflow#246)
Hi @sdll,
Awesome work here!
Kindly look into the same and I'll review the rest of the PR once fixed.
| CONVERTED_MODELS_DIR="efficientnet" | ||
| VIRTUALENV_DIR="venv" | ||
| MODEL_SOURCE_DIR="efficientnet-source" | ||
| MODEL_SOURCE_URL="https://github.com/sdll/efficientnet" |
There was a problem hiding this comment.
We should source the checkpoint from an official tfjs bucket and remove dependence on the forked repository
| }, | ||
| MEAN_RGB: [0.485, 0.456, 0.406], | ||
| STDDEV_RGB: [0.229, 0.224, 0.225], | ||
| IMAGENET_CLASSES: { |
There was a problem hiding this comment.
I think we should move these classes to a separate JSON file and import it here.
There was a problem hiding this comment.
This seems to be more natural, since otherwise we have to cast class labels to integers, losing the information stored in the data itself.
| @@ -0,0 +1,216 @@ | |||
| #!/usr/bin/env bash | |||
There was a problem hiding this comment.
This script throws the following error
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~ ............................. ~
~ Converting efficientnet-b0... ~
~ ............................. ~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "/home/manrajsingh/github/tfjs-models/efficientnet/dist/efficientnet-source/scripts/load_efficientnet.py", line 156, in <module>
import eval_ckpt_main
ModuleNotFoundError: No module named 'eval_ckpt_main'Kindly look into the same
There was a problem hiding this comment.
Recent updates to tensorflowjs_converter have introduced a bug which I could not fix for hours. I have tried to roll back (and forth!), but to no avail. The issue is explained here.
|
@sdll A few quick questions. Apart from the conversion issues (I'll try to reproduce that locally when I get a chance), I did have some issues running the build (incorrect reference to preamble) and running the demos (no keys in efficientnets variable thus non of the buttons work). I did hack it in, but didn't get a meaningful prediction. Are those parts still work in progress? At a higher level I was wondering how big the weights are (in MB)? And what kind of inference times you are getting? |
|
Sorry, @tafsiri, for the confusion, I glossed over changes I had to do following those based on @manrajgrover’s suggestions. This is fixed now, I have successfully published the module locally and tested each of the variants. Here is the screenshot:
Let me know if there are any issues other than those outlined here. Models quantized to 2 bytes weigh from 10.2 MB ( With regards to performance, I will refactor the code to follow the factory pattern of loading the model, taking inspiration from your review of DeepLab. Thanks for your time and help! |
Reviewable status: 0 of 1 approvals obtained (waiting on @manrajgrover and @sdll)
efficientnet/README.md, line 21 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
const efficientnet = new EfficientNet(model, isQuantized);
Done.
efficientnet/README.md, line 28 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
This should be a relative URL to the source file.
Done.
efficientnet/README.md, line 48 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
We should also pass
topKhere as an example.
Done.
efficientnet/README.md, line 58 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
EfficientNets rely on AutoML and compound scaling to achieve superior performance without compromising resource efficiency. The [AutoML Mobile framework](https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html) has helped develop a mobile-size baseline network, **EfficientNet-B0**, which is then improved by the compound scaling method to obtain EfficientNet-B1 to B7.
Done.
efficientnet/README.md, line 60 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
Let's format this html code
Done.
efficientnet/README.md, line 82 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
./scripts/convert_efficientnet.sh --target_dir=dist --use_venv=t
We will need to fix this after the issue with the converter is resolved (might need to switch to the docker-based pipeline to avoid issues on OS X, for example).
efficientnet/README.md, line 88 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
./scripts/convert_efficientnet.sh -h
Done.
efficientnet/demo/src/index.js, line 27 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
Let's add keys to Object dynamically during initialization.
Done.
efficientnet/scripts/convert_efficientnet.sh, line 145 at r1 (raw file):
Previously, manrajgrover (Manraj Singh) wrote…
We should source the checkpoint from an official tfjs bucket and remove dependence on the forked repository
Done.
aptlin
changed the title
Reviewable status: 0 of 1 approvals obtained (waiting on @manrajgrover and @sdll)
efficientnet/src/types.ts, line 25 at r5 (raw file):
|ImageData|HTMLImageElement|HTMLCanvasElement|HTMLVideoElement|tf.Tensor3D; // #TODO: Fix this after the issues with the converter are resolved
After the conversion issue is resolved, I will add support for all kinds of quantization.
|
@sdll just fyi i get the same issue with calls to Thanks for the model info |
|
@tafsiri, a fresh install did not show any issues: # cd tfjs-models/efficientnet
# git pull
cd demo
rm -rf node_modules/ .yalc/ .cache/
cd ..
yarn publish-local
cd demo
yarn
yarn link-local
yarn watchOne thing that I did notice though was that labels were off in Chrome due to the differences in CSS rendering:
This is fixed now:
|
This PR adds the implementation of EfficientNet B0-5 inference pipeline, with the support for quantized weights.
Roadmap
Translate predicted logits to top K classes
Test for memory leaks and validate inference
Add the demo
Improve the docs
Make a script converting original checkpoints to TF.js JSON
UPDATE 18/06: I have made a PR with the script to the most popular Keras implementation.
UPDATE 25/06:
The conversion tf.keras -> SavedModel -> TF.js Graph Model works well.UPDATE 08/07: The conversion to Keras in the first step turned out to be unnecessary, so I simplified the script, but could not handle a pesky bug with the converter (exporting to Keras first did not help as well, so the optimisation was not the source of the issue).
Integrate the TF Checkpoint to Keras converter script into the workflow for end-to-end porting of weights to TF.js JSON
UPDATE 18/06:
To make the model work, we need to implement two custom layers and an initializer.Fix the conversion issues
cc: @manrajgrover
This change is