Skip to content

Two small fixes for Flax examples#1597

Merged
marcvanzee merged 1 commit intomainfrom
test_399373571
Oct 4, 2021
Merged

Two small fixes for Flax examples#1597
marcvanzee merged 1 commit intomainfrom
test_399373571

Conversation

@copybara-service
Copy link

@copybara-service copybara-service bot commented Oct 4, 2021

Two small fixes in sst and vae

@google-cla google-cla bot added the cla: yes label Oct 4, 2021
@marcvanzee marcvanzee changed the title Implements a modular and easily extensible evaluation framework for both TFLite and TFjs. The evaluation framework has the following features: Two small fixes for Flax examples Oct 4, 2021
@marcvanzee marcvanzee requested a review from bastings October 4, 2021 09:37
@marcvanzee marcvanzee self-assigned this Oct 4, 2021
@copybara-service copybara-service bot changed the title Two small fixes for Flax examples Implements a modular and easily extensible evaluation framework for both TFLite and TFjs. The evaluation framework has the following features: Oct 4, 2021
@copybara-service copybara-service bot force-pushed the test_399373571 branch 2 times, most recently from eab674f to ce29692 Compare October 4, 2021 10:03
Copy link
Contributor

@bastings bastings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for sst2.

@codecov-commenter
Copy link

codecov-commenter commented Oct 4, 2021

Codecov Report

Merging #1597 (1820750) into main (68caa8a) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1597   +/-   ##
=======================================
  Coverage   82.75%   82.75%           
=======================================
  Files          68       68           
  Lines        5583     5583           
=======================================
  Hits         4620     4620           
  Misses        963      963           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68caa8a...1820750. Read the comment docs.

…oth TFLite and TFjs. The evaluation framework has the following features:

*  It is easy to add new Modules of examples since each Module is specified using a few lines of code (see `examples.py`).

* It is easy to add new converters since each converter is represented as a function (see `converters.py`). For instance, we could add the MLIR-based converter that the TFLite team is currently working on.

* The framework outputs a Markdown table (see `README.md`).

The framework has the following limitations:

* We only evaluate whether a Module converts, we do not compare any outputs between the converted model and the original model. This will require more effort, and it seems like we can do this as a follow-up if necessary (once a good fraction of ops are converted).

* If an example is missing multiple ops, then only the first missing op is reported. We could improve this by implementing mocked versions of non-working ops, which only output the right shapes. We could also consider doing this as a follow-up.

PiperOrigin-RevId: 399373571
@marcvanzee marcvanzee changed the title Implements a modular and easily extensible evaluation framework for both TFLite and TFjs. The evaluation framework has the following features: Two small fixes for Flax examples Oct 4, 2021
@marcvanzee marcvanzee merged commit ebaf281 into main Oct 4, 2021
@marcvanzee marcvanzee deleted the test_399373571 branch October 4, 2021 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants