No results when running sqlcoder34B-alpha

[SiChengliang]

Hi team and other friends,

There are no results when running sqlcoder34B-alpha model with sample code and database. I have checked pipe result, there is no relsult with beam_search = 5 setting. And if I set beam_search = 1, the performance is very poor and I believe it's not the model's performance.

Moreover, sqlcoder2 do have relsults, but should modify sample code as below.

Developed on 2*A100 40GB with no quantification
Any ideas with this issue, thanks a lot.

[SiChengliang]

I figure out why there are no relsult. for me, it's because the library version is too low, should update. finally, update the environment to

it has output.

[junzheng-wu-whitehorse]

Hey, I encountered into the same issue. I think its the keyword used to split the text is wrong. You should try to print the entire output and decide what splitter required.
Try modify the code in inference.py into something as:

    generated_query = pipe(
        prompt,
        num_return_sequences=1,
        eos_token_id=eos_token_id,
        pad_token_id=eos_token_id,
        )[0]["generated_text"]

[wongjingping]

Hi could you share the prompt that you used? The generated text contains the prompt so one would need to replace the final characters accordingly.

[ighoshsubho]

I would like to work on this issue. Can you assign it to me?

[rishsriv]

Hi there, this was fixed in #63. Closing for now – please feel free to reopen if you still run into issues.