GPTBigCode models generate noise results

[anxietymonger]

vllm/vllm/config.py

Line 101 in aa84c92

if (getattr(self.hf_config, "multi_query", False) and

Adding conditions for falcon models seems break inference for GPTBigCode models. It worked well on commit df5dd3c, but broke on commit aa84c92.

I have tried WizardCoder 15B, if I change the condition

if (getattr(self.hf_config, "multi_query", False) and
            (self.hf_config.model_type == "falcon" and
             not getattr(self.hf_config, "new_decoder_architecture", False))):

into

if getattr(self.hf_config, "multi_query", False):

, it works well.

I'm not sure whether this is a bug, or I just need some extra configuration?

[zhuohan123]

Fixed by #676. Thank you for reporting this!