Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Inference] Add support for large model via HDF5 model weight #82

Merged
merged 75 commits into from
Jul 8, 2021
Merged

[Inference] Add support for large model via HDF5 model weight #82

merged 75 commits into from
Jul 8, 2021

Conversation

xingyaoww
Copy link
Contributor

@xingyaoww xingyaoww commented Jun 30, 2021 •
edited
Loading

  • Support huggingface model export to HDF5 weights
  • Add support for loading HDF5 in inference module
  • Integrate HDF5 into CMakeLists
  • Update Dockerfile for HDF5 installation
  • Update build.md for HDF5 installation and build

Xingyao Wang added 30 commits June 23, 2021 15:05
add hdf5 as 3rdparty module
f91b04e
copy bart_export script for hdf5 export prototype
092eea0
update CMakeLists to include hdf5 for compilation
f544d95
support convert bart proto weight to hdf5
78e07db
remove old script
1b01817
rename hdf5 supporting script
55dab36
add print statement
f50d2c4
making protobuf as one parsing method;
58ea3a1 
add endswith utility function to determine the weight type;
done fix hdf5 compilation issue
aeedcad
done fix hdf5 compilation issue
456e53e
add hdf5 debugging code
8669d5d
Merge commit 'b3bf2f77dba1afb426db3936f00cae4fda268f8c' into h5-support
473db27
remove hdf5 as 3rdparty module
5b01663
fix build issue
2fcaf7a
fix endswith utility func typo
d05286c
got debugging script compiled and run as expected
0507cdb
add metadata to hdf5
922b24a
update bart export with missing fields
0f54fdc
update hdf5 dataset utility
1213f22
add error check for hdf5 dataset
3cfb827
drafted hdf5_get_config
638d44b
fix compilation error;
a6e1564 
commented unfinished part;
use wacky trick to get string saved correctly in hdf5
1ed8671
fixed segfault-causing string-related issue;
05cdd28 
add HDF5DatasetNotFoundError to handle is_multilingual special case;
fixed topk type issue
0ca876a
support read variable-length array from HDF5 into vector;
9e60915
finished hdf5_parse_emb_wei;
2e2b6b6
fixed raw_value in-place type issue;
64047dc
update build doc for hdf5
98ba5a5
fix lambda capture issue
a97670d
@Taka152 Taka152 linked an issue Jun 30, 2021 that may be closed by this pull request
Gpt exceeds maximum protobuf size of 2GB: 3096122166 #63
Closed
@xingyaoww xingyaoww marked this pull request as ready for review July 6, 2021 08:34
@xingyaoww xingyaoww changed the title [Inference Checkpoint] Add support for large model via HDF5 model weight [Inference] Add support for large model via HDF5 model weight Jul 6, 2021
Taka152
Taka152 previously approved these changes Jul 6, 2021
View reviewed changes
Copy link
Contributor

@Taka152 Taka152 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

examples/inference/python/utils.py Outdated Show resolved Hide resolved
@xingyaoww xingyaoww requested a review from Taka152 July 8, 2021 09:43
@Taka152 Taka152 merged commit 5ed3839 into bytedance:master Jul 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Taka152 Taka152 Taka152 approved these changes

@godweiyang godweiyang Awaiting requested review from godweiyang godweiyang is a code owner

@neopro12 neopro12 Awaiting requested review from neopro12 neopro12 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Gpt exceeds maximum protobuf size of 2GB: 3096122166
2 participants
@xingyaoww @Taka152