Fix crash in prepare_prompt

[NikolaBorisov]

max() is being called with empty list of arguments. Somehow there are no sequnces

[NikolaBorisov]

Trying to fix #2476

[NikolaBorisov]

@simon-mo I'm not sure this will fix it, but number of different models running in vllm are currently crashing with #2476

[simon-mo]

I think the cause must be deeper on why would the input list be empty.

[NikolaBorisov]

There is assert assert len(seq_group_metadata_list) > 0. So the outer list is not empty, but the inner list could be empty.

Here is the code in schedule. seq_group.get_seqs(status=SequenceStatus.RUNNING): could return empty. We could add some code there to skip the sequence if it is empty.

        for seq_group in scheduler_outputs.scheduled_seq_groups:
            seq_data: Dict[int, SequenceData] = {}
            block_tables: Dict[int, List[int]] = {}
            for seq in seq_group.get_seqs(status=SequenceStatus.RUNNING):
                seq_id = seq.seq_id
                seq_data[seq_id] = seq.data
                block_tables[seq_id] = self.block_manager.get_block_table(seq)

            seq_group_metadata = SequenceGroupMetadata(
                request_id=seq_group.request_id,
                is_prompt=scheduler_outputs.prompt_run,
                seq_data=seq_data,
                sampling_params=seq_group.sampling_params,
                block_tables=block_tables,
                prefix=seq_group.prefix,
            )
            seq_group_metadata_list.append(seq_group_metadata)

[simon-mo]

Not sure whether this is the bug #2463 fixed

[NikolaBorisov]

Seems likely. Let me try to test if this fixes it

[simon-mo]

@NikolaBorisov does #2463 fixes it?

[simon-mo]

I'm closing this assuming it is fixed. Please feel free to reopen.

[NikolaBorisov]

@simon-mo Sorry for the late response. Yes, this problem looks fixed. But I see another issue: #2570