how to export the model #41
Comments
|
You can export the models trained by neurst using the scripts in https://github.com/godweiyang/chatbot/tree/master/export. The following is the export command: python3 export.py \
--model_dir models \
--output_file model.pb \
--generation_method topk \
--topk 4 \
--length_penalty 0.6 \
--beam_size 4 |
|
@godweiyang Traceback (most recent call last):
File "export.py", line 161, in <module>
app.run(_main, argv=["pseudo.py"])
File "/mnt/data_0301_12t/xingchaolong/home/pyenvs/env_neurst_lightseq/lib/python3.8/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/mnt/data_0301_12t/xingchaolong/home/pyenvs/env_neurst_lightseq/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "export.py", line 154, in _main
export_lightseq_model(
File "export.py", line 127, in export_lightseq_model
transfer_ckpt2pb(
File "/mnt/data_0301_12t/xingchaolong/home/codes/trans/neurst/demo/export/transformer_proto_utils.py", line 256, in transfer_ckpt2pb
_fill_layer(
File "/mnt/data_0301_12t/xingchaolong/home/codes/trans/neurst/demo/export/transformer_proto_utils.py", line 139, in _fill_layer
assert len(tmp) == 1
AssertionErrorthe cmd as below python3 export.py --model_dir ../wmt14_en_de/benchmark_base/ --output_file ../infer/model.pb --beam_size 4 --length_penalty 0.6and the model is Any help for this？ |
Can you provide your trainning command and neurst version? |
|
@godweiyang git clone -b lightseq https://github.com/bytedance/neurst.git
cd neurst/
pip3 install -e .
pip3 install lightseq-tf2.4.1-cuda11.0.221==2.0.1
pip3 install absl-py
pip3 install tensorflow==2.4.1
python3 -m neurst.cli.run_exp --config_paths wmt14_en_de/training_args.yml,wmt14_en_de/translation_bpe.yml,wmt14_en_de/validation_args.yml --hparams_set transformer_base --model_dir wmt14_en_de/benchmark_base
entry.class: trainer
entry.params:
train_steps: 200000
summary_steps: 200
save_checkpoint_steps: 1000
criterion.class: label_smoothed_cross_entropy
criterion.params:
label_smoothing: 0.1
dataset.class: ParallelTextDataset
dataset.params:
src_file: wmt14_en_de/train.en.tok.bpe.txt
trg_file: wmt14_en_de/train.de.tok.bpe.txt
data_is_processed: True
task.class: translation
task.params:
batch_by_tokens: True
batch_size: 4096
max_src_len: 128
max_trg_len: 128
src_data_pipeline.class: TextDataPipeline
src_data_pipeline.params:
language: en
tokenizer: moses
subtokenizer: bpe
subtokenizer_codes: wmt14_en_de/codes.bpe
vocab_path: wmt14_en_de/vocab.en
trg_data_pipeline.class: TextDataPipeline
trg_data_pipeline.params:
language: de
tokenizer: moses
subtokenizer: bpe
subtokenizer_codes: wmt14_en_de/codes.bpe
vocab_path: wmt14_en_de/vocab.de
validator.class: SeqGenerationValidator
validator.params:
eval_dataset: ParallelTextDataset
eval_dataset.params:
src_file: wmt14_en_de/newstest2013.en.txt
trg_file: wmt14_en_de/newstest2013.de.txt
eval_batch_size: 64
eval_start_at: 5000
eval_steps: 1000
eval_criterion: label_smoothed_cross_entropy
eval_search_method: beam_search
eval_search_method.params:
beam_size: 4
length_penalty: 0.6
maximum_decode_length: 160
extra_decode_length: 50
eval_metric: bleu
eval_top_checkpoints_to_keep: 10
eval_auto_average_checkpoints: True |
Can you undo the comment in |
|
@godweiyang 2021-12-27 15:27:32.058066: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I1227 15:27:35.874068 139747470812992 flags_core.py:493] ==========================================================================
I1227 15:27:35.874294 139747470812992 flags_core.py:494] Parsed all matched flags:
I1227 15:27:35.874423 139747470812992 flags_core.py:499] model_dir: ../wmt14_en_de/benchmark_base/ # (default: None) The path to the well-trained checkpoint.
I1227 15:27:35.874494 139747470812992 flags_core.py:499] output_file: ../infer/model.pb # (default: None) The path to saving the extracted weights.
I1227 15:27:35.874563 139747470812992 flags_core.py:499] generation_method: beam_search # (default: beam_search) The generation method: beam_search (default), topk or topp.
I1227 15:27:35.874632 139747470812992 flags_core.py:499] beam_size: 4 # (default: None) The beam width of sequence generation for Transformer model.
I1227 15:27:35.874716 139747470812992 flags_core.py:499] length_penalty: 0.6 # (default: -1) The length penalty of sequence generation for Transformer model.
I1227 15:27:35.874784 139747470812992 flags_core.py:499] max_seq_len: 256 # (default: 256) The maximum sequence length.
I1227 15:27:35.874855 139747470812992 flags_core.py:499] extra_decode_length: 50 # (default: 50) The extra decoding length of sequence generation for Transformer model.
I1227 15:27:35.874926 139747470812992 flags_core.py:499] topk: 4 # (default: 4) topk for sampling, only 1,2,4,...,32 are valid
I1227 15:27:35.874995 139747470812992 flags_core.py:499] topp: 0.75 # (default: 0.75) topp for sampling
I1227 15:27:35.875064 139747470812992 flags_core.py:499] use_aligned_penalty: False # (default: False) Whether to use aligned length penalty
I1227 15:27:35.875136 139747470812992 flags_core.py:499] align_len_ratio: 1.0 # (default: 1.0) The length ratio against source sequence
I1227 15:27:35.875207 139747470812992 flags_core.py:499] align_len_bias: 3.0 # (default: 3.0) The length bias
I1227 15:27:35.875275 139747470812992 flags_core.py:499] align_len_alpha: 1.0 # (default: 1.0) The exponent of length penalty
I1227 15:27:35.875342 139747470812992 flags_core.py:524]
I1227 15:27:35.875405 139747470812992 flags_core.py:525] Other flags:
I1227 15:27:35.875470 139747470812992 flags_core.py:527] config_paths: None
I1227 15:27:35.875535 139747470812992 flags_core.py:542] ==========================================================================
I1227 15:27:35.875828 139747470812992 export.py:116] Extracting weights from ../wmt14_en_de/benchmark_base/ to ../infer/model.pb
I1227 15:27:35.891601 139747470812992 configurable.py:312] Loading models configs from ../wmt14_en_de/benchmark_base/model_configs.yml
I1227 15:27:35.891697 139747470812992 export.py:120] Extract and export Transformer weights using method: beam_search.
I1227 15:27:35.891793 139747470812992 transformer_proto_utils.py:234] Using /mnt/data_0301_12t/xingchaolong/home/codes/trans/neurst/demo/export/transformer.proto
self_attention_prepost_wrapper gamma
[]
Traceback (most recent call last):
File "export.py", line 161, in <module>
app.run(_main, argv=["pseudo.py"])
File "/mnt/data_0301_12t/xingchaolong/home/pyenvs/env_neurst_lightseq/lib/python3.8/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/mnt/data_0301_12t/xingchaolong/home/pyenvs/env_neurst_lightseq/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "export.py", line 154, in _main
export_lightseq_model(
File "export.py", line 127, in export_lightseq_model
transfer_ckpt2pb(
File "/mnt/data_0301_12t/xingchaolong/home/codes/trans/neurst/demo/export/transformer_proto_utils.py", line 256, in transfer_ckpt2pb
_fill_layer(
File "/mnt/data_0301_12t/xingchaolong/home/codes/trans/neurst/demo/export/transformer_proto_utils.py", line 139, in _fill_layer
assert len(tmp) == 1
AssertionError |
I know. DO NOT use lightseq branch of neurst, otherwise the varibles can not be exported. So use master branch to train the models. |
|
@godweiyang OK, I will test later. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
How to export model with the format used by lightseq and tortion?
The text was updated successfully, but these errors were encountered: