Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce binary size for Linux and Mac package #3337

Merged
merged 26 commits into from
Mar 25, 2021
Merged

Conversation

jcwchen
Copy link
Member

@jcwchen jcwchen commented Mar 16, 2021

Description

  • Instead of building C++ protobuf with default configuration, release pipelines should build it with release mode. It can remarkably reduce binary size for Linux package and slightly reduce for Mac package.

Linux package -- 18.9MB ->11.5MB
Mac package -- 12MB -> 11.3MB

Meanwhile; the total running time will be much faster as well: (Linux 59mins -> 8 mins; Mac 26mins -> 9mins)

  • Deprecate Python 3.5 to prevent failure caused by Python 3.5 incompatibility

Motivation
ONNX 1.8.1 PyPI package becomes much larger than 1.8.0, especially Linux package. The main reason is ONNX upgraded its protobuf-compiler version from 2.6 to 3.11. We need to figure out a reasonable way to shrink protobuf dependency even though it needs to be upgraded.

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen requested a review from a team as a code owner March 16, 2021 23:07
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@@ -4,7 +4,7 @@ on:
push:
branches: [master, rel-*]
pull_request:
branches: [rel-*]
branches: [rel-*, master] # TODO: remove it before merge

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mac CI fails because of this pypa/setuptools-scm#542. Will try to reduce binary size as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ONNX decided to deprecate python 3.5 in the following 1.9. Will remove it in this PR to prevent failure caused by Python 3.5 incompatibility

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen changed the title [WIP] Reduce binary size for Linux package Reduce binary size for Linux and Mac package Mar 18, 2021
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@askhade askhade added this to the 1.9 milestone Mar 18, 2021
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen jcwchen changed the title Reduce binary size for Linux and Mac package Reduce binary size for Linux and Mac package; Deprecate Python 3.5 Mar 19, 2021
@jcwchen jcwchen linked an issue Mar 22, 2021 that may be closed by this pull request
@@ -39,6 +39,8 @@ jobs:
- name: Install Python dependencies
run: |
python -m pip install -q --upgrade pip
# Latest setuptools-scm-6.0.1 (dependency of setuptools) does not support Python 3.5 anymore
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still relevant if we want to remove support for Python 3.5?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Removed. Thanks

@askhade
Copy link
Contributor

askhade commented Mar 22, 2021

can we separate this in 2 PRs... to keep things clean.... Create a new 1 for python deprecation and keep this for binary size reduction. Thanks!

@jcwchen jcwchen changed the title Reduce binary size for Linux and Mac package; Deprecate Python 3.5 Reduce binary size for Linux and Mac package Mar 22, 2021
Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>
@jcwchen
Copy link
Member Author

jcwchen commented Mar 22, 2021

can we separate this in 2 PRs... to keep things clean.... Create a new 1 for python deprecation and keep this for binary size reduction. Thanks!

Sure thing. A new PR (#3353) decouple from this PR

@askhade askhade merged commit b51f033 into onnx:master Mar 25, 2021
@jcwchen jcwchen deleted the reduce-linux branch September 6, 2022 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants