Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize binary size #5820

Merged
merged 7 commits into from Nov 12, 2019
Merged

Optimize binary size #5820

merged 7 commits into from Nov 12, 2019

Conversation

@Hoverbear
Copy link
Contributor

Hoverbear commented Nov 5, 2019

What have you changed?

This is a 3/4 fix for #5303. dwz is still an issue for tikv-server and that is left up to a future PR.

What is the type of the changes?

Improvement.

How is the PR tested?

See #5303 (comment)

Does this PR affect documentation (docs) or should it be mentioned in the release notes?

Nope

Does this PR affect tidb-ansible?

Nope

Refer to a related PR or issue link (optional)

#5303

Benchmark result if necessary (optional)

Any examples? (optional)

#5303 (comment)

Signed-off-by: Ana Hobden <operator@hoverbear.org>
@Hoverbear Hoverbear requested review from siddontang and zhouqiang-cl Nov 5, 2019
Makefile Outdated
# Reduce binary size by compressing binaries.
# Currently errors with `Couldn't find DIE referenced by DW_AT_abstract_origin`
# dwz ${CARGO_TARGET_DIR}/release/tikv-server
dwz ${CARGO_TARGET_DIR}/release/tikv-ctl

This comment has been minimized.

Copy link
@siddontang

siddontang Nov 6, 2019

Contributor

em, why tikv-ctl can use dwz but tikv-server can't?

This comment has been minimized.

Copy link
@Hoverbear

Hoverbear Nov 6, 2019

Author Contributor

I am not sure!

This comment has been minimized.

Copy link
@zhouqiang-cl

zhouqiang-cl Nov 6, 2019

Member

seems tikv-ctl is already small

This comment has been minimized.

Copy link
@Hoverbear

Hoverbear Nov 6, 2019

Author Contributor

Agreed :-P

This comment has been minimized.

Copy link
@Hoverbear

Hoverbear Nov 6, 2019

Author Contributor
make[2]: Leaving directory '/home/hoverbear/git/tikv'
# Reduce binary size by compressing binaries.
# Currently errors with `Couldn't find DIE referenced by DW_AT_abstract_origin`
dwz /home/hoverbear/git/tikv/target/release/tikv-server
dwz: /home/hoverbear/git/tikv/target/release/tikv-server: Couldn't find DIE referenced by DW_AT_abstract_origin
make[1]: *** [Makefile:160: build_dist_release] Error 1
make[1]: Leaving directory '/home/hoverbear/git/tikv'
make: *** [Makefile:149: dist_release] Error 2
@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Nov 6, 2019

@glorv PTAL too

Makefile Outdated
# Currently errors with `Couldn't find DIE referenced by DW_AT_abstract_origin`
# dwz ${CARGO_TARGET_DIR}/release/tikv-server
dwz ${CARGO_TARGET_DIR}/release/tikv-ctl
objcopy --compress-debug-sections=zlib ${CARGO_TARGET_DIR}/release/tikv-server

This comment has been minimized.

Copy link
@glorv

glorv Nov 6, 2019

Collaborator

I test on a centos localhost, It seems compress by --compress-debug-sections=zlib, when I gdb attach tikv-server the backtrace lost source code line number info in compare to --compress-debug-sections=zlib-gnu, so why we choose zlib instead of zlib-gnu

This comment has been minimized.

Copy link
@Hoverbear

Hoverbear Nov 6, 2019

Author Contributor

Whoa! Nice catch!

@Hoverbear

This comment has been minimized.

Copy link
Contributor Author

Hoverbear commented Nov 6, 2019

@glorv Were you able to run dwz on tikv-server?

@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 6, 2019

I ran the sysbench_oltp tests suit last night, with the tikv-server compress by objcopy --compress-debug-sections=zlib-gnu bin/tikv-server here are the result:

Version:
   tidb: ea4cafa547aab1da02be94a5e7eeedb89a5c0e08(master) 2019-11-05 11:27:13
   tikv: a2b85ba0983d96f887c8ddcdf224eb38b69d0eee(master) 2019-11-05 11:28:01
   pd: a1a7b50f42863abcc958680124624197e590c064(master) 2019-11-05 10:18:50

Origin Test Result:

test-1: < oltp_update_non_index >
   * QPS : 32860.35 ± 0.3892% (std=104.56)
   * TPS : 32860.35 ± 0.3892% (std=104.56)
   * AvgMs : 7.79 ± 0.4109% (std=0.02)
   * PercentileMs99 : 20.74 ± 0.0000% (std=0.00) 
           
test-2: < oltp_insert >
   * QPS : 23663.58 ± 2.0913% (std=274.67) 
   * TPS : 23663.58 ± 2.0913% (std=274.67)
   * AvgMs : 10.81 ± 2.1088% (std=0.13)
   * PercentileMs99 : 22.05 ± 1.0614% (std=0.19)
           
test-3: < oltp_read_write >
   * QPS : 41876.49 ± 0.2870% (std=96.11) 
   * TPS : 2084.09 ± 0.2873% (std=4.89)
   * AvgMs : 122.75 ± 0.2949% (std=0.29) 
   * PercentileMs99 : 244.38 ± 0.0000% (std=0.00)
           
test-4: < oltp_write_only >
   * QPS : 39632.57 ± 0.2493% (std=67.26)
   * TPS : 6586.59 ± 0.2535% (std=11.45)
   * AvgMs : 38.67 ± 0.8119% (std=0.22) 
   * PercentileMs99 : 77.19 ± 0.0000% (std=0.00)
           
test-5: < oltp_point_select >
   * QPS : 139655.31 ± 3.5632% (std=2871.35) 
   * TPS : 139655.31 ± 3.5632% (std=2871.35) 
   * AvgMs : 1.83 ± 3.7118% (std=0.04) 
   * PercentileMs99 : 6.16 ± 2.5974% (std=0.09)

Test Result With tikv-server compressed by objcopy

test-1: < oltp_update_non_index >
   * QPS : 32668.18 ± 1.8862% (std=344.75) delta: -0.58% (p=0.612)
   * TPS : 32668.18 ± 1.8862% (std=344.75) delta: -0.58%
   * AvgMs : 7.83 ± 1.8637% (std=0.08) delta: 0.51%
   * PercentileMs99 : 20.74 ± 0.0000% (std=0.00) delta: 0.00%
           
test-2: < oltp_insert >
   * QPS : 23631.62 ± 1.6345% (std=231.33) delta: -0.13% (p=0.631)
   * TPS : 23631.62 ± 1.6345% (std=231.33) delta: -0.13%
   * AvgMs : 10.79 ± 0.8809% (std=0.07) delta: -0.19%
   * PercentileMs99 : 22.13 ± 2.5398% (std=0.32) delta: 0.36%
           
test-3: < oltp_read_write >
   * QPS : 41803.93 ± 0.3225% (std=129.39) delta: -0.17% (p=0.280)
   * TPS : 2080.57 ± 0.3203% (std=6.41) delta: -0.17%
   * AvgMs : 122.86 ± 0.4200% (std=0.29) delta: -0.09%
   * PercentileMs99 : 242.64 ± 1.0782% (std=2.14) delta: -0.71%
           
test-4: < oltp_write_only >
   * QPS : 39820.40 ± 0.3794% (std=112.47) delta: 0.47% (p=0.628)
   * TPS : 6617.62 ± 0.3838% (std=18.95) delta: 0.47%
   * AvgMs : 38.48 ± 1.2267% (std=0.32) delta: -0.49%
   * PercentileMs99 : 77.19 ± 0.0000% (std=0.00) delta: 0.00%
           
test-5: < oltp_point_select >
   * QPS : 140916.51 ± 1.0335% (std=1094.40) delta: 0.90% (p=0.464)
   * TPS : 140916.51 ± 1.0335% (std=1094.40) delta: 0.90%
   * AvgMs : 1.81 ± 0.8264% (std=0.01) delta: -1.09%
   * PercentileMs99 : 6.14 ± 2.4748% (std=0.09) delta: -0.32%

@Hoverbear @zhouqiang-cl

@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 6, 2019

@glorv Were you able to run dwz on tikv-server?

sadly no, the dwz command also return Couldn't find DIE referenced by DW_AT_abstract_origin error @Hoverbear

@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 6, 2019

/run-all-tests

Copy link
Member

zhouqiang-cl left a comment

LGTM

@Hoverbear Hoverbear self-assigned this Nov 6, 2019
@Hoverbear Hoverbear added the C: Build label Nov 6, 2019
Signed-off-by: Ana Hobden <operator@hoverbear.org>
@Hoverbear

This comment has been minimized.

Copy link
Contributor Author

Hoverbear commented Nov 6, 2019

@glorv I also see this error on some distros (Ubuntu 19.10 here):

dwz /home/hoverbear/git/tikv/target/release/tikv-ctl
dwz: /build/dwz-25Zi9f/dwz-0.13/dwz.c:9584: recompute_abbrevs: Assertion `off == cu_size' failed.

Looks related to https://bugzilla.redhat.com/show_bug.cgi?id=1456261#c17, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892828, which lead to https://sourceware.org/bugzilla/show_bug.cgi?id=24764

@Hoverbear Hoverbear force-pushed the Hoverbear:optimize-binary-size branch from 7cfb169 to 37f0c11 Nov 6, 2019
Signed-off-by: Ana Hobden <operator@hoverbear.org>
@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 8, 2019

LGTM

@Hoverbear Hoverbear requested a review from zhouqiang-cl Nov 8, 2019
@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Nov 11, 2019

/test

2 similar comments
@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Nov 11, 2019

/test

@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 11, 2019

/test

@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Nov 11, 2019

/merge

@sre-bot sre-bot added the S: CanMerge label Nov 11, 2019
Copy link
Member

zhouqiang-cl left a comment

LGTM

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 11, 2019

/run-all-tests

@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Nov 11, 2019

@glorv PTAL again

@glorv

This comment has been minimized.

Copy link
Collaborator

glorv commented Nov 11, 2019

LGTM

@glorv
glorv approved these changes Nov 11, 2019
Copy link
Contributor

mahjonp left a comment

LGTM

@zhouqiang-cl zhouqiang-cl merged commit 5bebd6a into tikv:master Nov 12, 2019
6 checks passed
6 checks passed
DCO All commits are signed off!
Details
idc-jenkins-ci-tikv/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-copr-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
FateTHarlaown added a commit to FateTHarlaown/tikv that referenced this pull request Nov 12, 2019
* Optimize binary size

Signed-off-by: Ana Hobden <operator@hoverbear.org>
hawkingrei added a commit to hawkingrei/tikv that referenced this pull request Dec 1, 2019
* Optimize binary size

Signed-off-by: Ana Hobden <operator@hoverbear.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.