[llava][15/N] Extract out text decoder runner #4567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

facebook-github-bot merged 9 commits into gh/larryliu0820/44/base from gh/larryliu0820/44/head

Aug 9, 2024

Contributor

larryliu0820 commented Aug 6, 2024 •

edited

Loading

Stack from ghstack (oldest at bottom):

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new TextDecoderRunner provides 2 APIs:

step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.
logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: D60856571


          [llava][15/N] Extract out text decoder runner

d963403

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

[ghstack-poisoned]

larryliu0820 mentioned this pull request

[llava][14/N] Refactor runner prefill() and run_model_step() #4556

Merged

pytorch-bot bot commented Aug 6, 2024 •

edited

Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4567

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e8da443 with merge base 92edd04 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot added the CLA Signed label

larryliu0820 added a commit that referenced this pull request


          [llava][15/N] Extract out text decoder runner

53a5494

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

ghstack-source-id: 72cdfbc
Pull Request resolved: #4567

Contributor Author

larryliu0820 commented Aug 6, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "[llava][15/N] Extract out text decoder runner"

7b3c6d8

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

larryliu0820 added a commit that referenced this pull request


          [llava][15/N] Extract out text decoder runner

ae65051

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

ghstack-source-id: 2628a3e
Pull Request resolved: #4567

Contributor Author

larryliu0820 commented Aug 6, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

JacobSzwejbka approved these changes

View reviewed changes


          Update on "[llava][15/N] Extract out text decoder runner"

a1e6c5e

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

larryliu0820 mentioned this pull request

[llava][16/N] Extract out prefill logic into a new class #4585

Merged

Contributor Author

larryliu0820 commented Aug 7, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "[llava][15/N] Extract out text decoder runner"

de38ad7

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

Contributor Author

larryliu0820 commented Aug 7, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

larryliu0820 added 2 commits

August 7, 2024 15:30


          Update on "[llava][15/N] Extract out text decoder runner"

46fbce1

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]


          Update on "[llava][15/N] Extract out text decoder runner"

94903ed

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

larryliu0820 mentioned this pull request

[llava][17/N] Move util.h into /e/llm/runner #4588

Merged

Contributor Author

larryliu0820 commented Aug 7, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "[llava][15/N] Extract out text decoder runner"

0cf1345

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

Contributor Author

larryliu0820 commented Aug 7, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "[llava][15/N] Extract out text decoder runner"

ad1647d

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

Contributor Author

larryliu0820 commented Aug 8, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.


          Update on "[llava][15/N] Extract out text decoder runner"

e8da443

Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.

This new `TextDecoderRunner` provides 2 APIs:

* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.

* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.

Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)

[ghstack-poisoned]

Contributor Author

larryliu0820 commented Aug 8, 2024

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot merged commit 30c3a15 into gh/larryliu0820/44/base

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels