Skip to content

Add support for aquila #663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Aug 22, 2023
Merged

Add support for aquila #663

merged 10 commits into from
Aug 22, 2023

Conversation

shunxing1234
Copy link
Contributor

No description provided.

ftgreat and others added 4 commits August 3, 2023 06:28
@ftgreat
add aquila
db0f022 
Signed-off-by: ftgreat <ftgreat@163.com>
@shunxing1234
Merge branch 'main' into main
5de8474
@shunxing1234
fix some bug
3f20c72 
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
@shunxing1234
delete pdb
967b55e 
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
@shunxing1234
Copy link
Contributor Author

In order to merge this PR, do I need to provide any additional information?

@zhuohan123
Copy link
Member

@shunxing1234 Thank you for your great contribution! Can you make sure you pass the format checks and give one example that this model is outputting correct results?

shunxing1234 and others added 3 commits August 9, 2023 13:41
@shunxing1234
Merge branch 'main' into main
9813a2b
@shunxing1234
fix bugs
47d3403 
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
@shunxing1234
fix bugs
7b5120f 
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
@shunxing1234
Copy link
Contributor Author

shunxing1234 commented Aug 9, 2023 •
edited
Loading

@shunxing1234 Thank you for your great contribution! Can you make sure you pass the format checks and give one example that this model is outputting correct results?

https://github.com/FlagAI-Open/FlagAI/tree/master/examples/Aquila
Hi, thanks for your reply. Here is the outputting of aquila
image

@gesanqiu
Copy link
Contributor

gesanqiu commented Aug 9, 2023 •
edited
Loading

I have been verified the greedy output of transforemers and vLLM on BAAI/AquilaChat-7B last week, they are the same. What needs pay attention to is Aquila has several stop str, such as '###', '', '[UNK]', and I just made a PR to FastChat, after merged we can use openai_api.py on BAAI/AquilaChat-7B.

@shunxing1234
Copy link
Contributor Author

I have been verified the greedy output of transforemers and vLLM on BAAI/AquilaChat-7B last week, they are the same. What needs pay attention to is Aquila has several stop str, such as '###', '', '[UNK]', and I just made a PR to FastChat, after merged we can use openai_api.py on BAAI/AquilaChat-7B.

@gesanqiu Thanks for your Contribution!

shunxing1234 and others added 3 commits August 10, 2023 01:13
@shunxing1234
delete whitespace
a5730d9 
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
@zhuohan123
format
88e3363
@zhuohan123
fix order
336a2b2
zhuohan123
zhuohan123 approved these changes Aug 22, 2023
View reviewed changes
Copy link
Member

@zhuohan123 zhuohan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just tested out BAAI/AquilaChat-7B and it works great on a single A100. Thank you for your contribution!

@zhuohan123 zhuohan123 merged commit ad5f2fe into vllm-project:main Aug 22, 2023
@ftgreat
Copy link
Contributor

ftgreat commented Aug 23, 2023

@zhuohan123 one typo pr #836 , please review it. thanks.

randxie pushed a commit to randxie/vllm that referenced this pull request Aug 29, 2023
@shunxing1234 @ftgreat @randxie
Add support for aquila (vllm-project#663)
09d7ca5 
* add aquila

Signed-off-by: ftgreat <ftgreat@163.com>

* fix some bug

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete pdb

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete whitespace

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* format

* fix order

---------

Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>
hongxiayang pushed a commit to hongxiayang/vllm that referenced this pull request Feb 13, 2024
@shunxing1234 @ftgreat
Add support for aquila (vllm-project#663)
a6cda78 
* add aquila

Signed-off-by: ftgreat <ftgreat@163.com>

* fix some bug

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete pdb

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete whitespace

Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* format

* fix order

---------

Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>
yma11 pushed a commit to yma11/vllm that referenced this pull request Jan 23, 2025
@mfylcek
Fix for random sampler recompilations for incomplete batches (vllm-pr…
e977f2a 
…oject#663)

Changes the sampler used by dummy sequences to greedy if any
sequence is using it. Prevents sampler recompilations.
amy-why-3459 pushed a commit to amy-why-3459/vllm that referenced this pull request Sep 15, 2025
@Yikun
Remove prompt string from engine core data structures (vllm-project#663)
742f679 
### What this PR does / why we need it?
vLLM Ascend side followup on:
[Core] Remove prompt string from engine core data structures

vllm-project@df6f3ce

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
CI passed

Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhuohan123 zhuohan123 zhuohan123 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@shunxing1234 @zhuohan123 @gesanqiu @ftgreat