GPT2 with one step beam search

[xi-liu-ds]

Description: In this PR we add support for one-step beam search in ONNX graph.

Motivation and Context

• Language models (e.g. GPT) usually need beam search to decode output token sequences. This PR will fulfill this purpose by adding GPT2LMHeadModel_BeamSearchStep class and one-step beam search included directly into ONNX compute graph.

How to run
In bash/CMD/PowerShell, run:

python onnxruntime/python/tools/transformers/convert_to_onnx.py \
--model_name_or_path=[path/to/model/folder] --model_class=GPT2LMHeadModel_BeamSearchStep --output=[path/to/output/onnx_file_name] -o --precision=int8

and optionally with --input_test_file=[path/to/test/file]

[hariharans29]

Hi @xi-liu-ds - Thanks for the PR and contribution. Do you have some time to address the comments in the PR ?

[hariharans29]

/azp run Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, centos7_cpu, centos7_cpu (linux_centos_ci Debug), centos7_cpu (linux_centos_ci Release), orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-amd-gpu-ci-pipeline, Linux Nuphar CI Pipeline, orttraining-distributed

[hariharans29]

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux CPU x64 NoContribops CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, MacOS NoContribops CI Pipeline, Windows CPU CI Pipeline, orttraining-ortmodule, orttraining-ortmodule-distributed

[hariharans29]

/azp run Linux GPU CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 6 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 10 pipeline(s).

[xi-liu-ds]

Hi @xi-liu-ds - Thanks for the PR and contribution. Do you have some time to address the comments in the PR ?

Hi @hariharans29 sorry I was busy with another task. I am working on addressing the comments right now.

BTW, I am wondering can I kick off all pipelines myself or I have to go through you guys?

Thanks!

[hariharans29]

Hi @xi-liu-ds - Thanks for the PR and contribution. Do you have some time to address the comments in the PR ?

Hi @hariharans29 sorry I was busy with another task. I am working on addressing the comments right now.

BTW, I am wondering can I kick off all pipelines myself or I have to go through you guys?

Thanks!

Thank you for addressing the comments. We will have to kick off pipelines for you. Let me know when you are done with addressing the comments and I ll kick them off. Thanks.

[xi-liu-ds]

Hi @xi-liu-ds - Thanks for the PR and contribution. Do you have some time to address the comments in the PR ?

Hi @hariharans29 sorry I was busy with another task. I am working on addressing the comments right now.
BTW, I am wondering can I kick off all pipelines myself or I have to go through you guys?
Thanks!

Thank you for addressing the comments. We will have to kick off pipelines for you. Let me know when you are done with addressing the comments and I ll kick them off. Thanks.

Hi @tianleiwu @hariharans29 I have addressed all the comments. Now it is ready for another round of review. @hariharans29 Could you help kick off pipelines? Thank you all!