Commit 24eb005
committed
Update base for Update on "[llava][15/N] Extract out text decoder runner"
Last PR #4556 refactored run_model_step() so that it is suitable to be
extracted out as a separate class.
This new `TextDecoderRunner` provides 2 APIs:
* step(tokens, start_pos)
This API takes one or more tokens with start_pos and feed them into Module. Return a tensor of logits.
* logits_to_token(logits)
This API samples the result and returns a token. We don't expect this
logic to change across different runners.
Differential Revision: [D60856571](https://our.internmc.facebook.com/intern/diff/D60856571)
[ghstack-poisoned]1 parent 306b656 commit 24eb005
1 file changed
+4
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
155 | | - | |
| 155 | + | |
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| |||
164 | 164 | | |
165 | 165 | | |
166 | 166 | | |
| 167 | + | |
167 | 168 | | |
168 | 169 | | |
169 | 170 | | |
| |||
177 | 178 | | |
178 | 179 | | |
179 | 180 | | |
| 181 | + | |
180 | 182 | | |
181 | 183 | | |
182 | 184 | | |
| |||
188 | 190 | | |
189 | 191 | | |
190 | 192 | | |
191 | | - | |
| 193 | + | |
192 | 194 | | |
193 | 195 | | |
194 | 196 | | |
| |||
0 commit comments