Is origin_input_ids expected to always be array.array in SGLang v0.5.13+?

[Galmut]

Before describing the issue, I'd like to note that I couldn't test this with the latest SGLang because of version compatibility issues with the current SMG gRPC stack. The investigation below is therefore based on SGLang v0.5.14.

While integrating SGLang with SGLang Model Gateway (SMG) using smg-grpc-servicer, I found what appears to be a type mismatch related to changes introduced around SGLang v0.5.13.

Environment

• SGLang: 0.5.14
• smg-grpc-servicer: 0.6.0
• smg-grpc-proto: 0.4.14

Problem

When starting SGLang through the SMG gRPC integration, the scheduler fails with:

TypeError: can only concatenate list (not "array.array") to list

The traceback points to:

python/sglang/srt/managers/schedule_batch.py

Specifically:

self.full_untruncated_fill_ids = self.origin_input_ids + self.output_ids

At runtime:

type(self.origin_input_ids) == list
type(self.output_ids) == array.array

which causes the exception.

Investigation

From what I found:

SGLang v0.5.12

output_ids was initialized as:

self.output_ids = []

SGLang v0.5.13+

It was changed to:

self.output_ids = array("q")

However, in Req.__init__() (python/sglang/srt/managers/schedule_batch.py), origin_input_ids is still stored without type normalization:

self.origin_input_ids = origin_input_ids

If origin_input_ids is passed as a Python list, later scheduler code assumes it has the same type as output_ids, resulting in the exception above.

Temporary workaround

On my side, converting origin_input_ids to array("q") in Req.__init__() avoids the scheduler crash.

For example:

- self.origin_input_ids = origin_input_ids
+ if not isinstance(origin_input_ids, array):
+     origin_input_ids = array("q", origin_input_ids)
+ self.origin_input_ids = origin_input_ids

- self.origin_input_ids_unpadded = (
-     origin_input_ids_unpadded
-     if origin_input_ids_unpadded
-     else self.origin_input_ids
- )
+ if origin_input_ids_unpadded is not None and not isinstance(origin_input_ids_unpadded, array):
+     origin_input_ids_unpadded = array("q", origin_input_ids_unpadded)

+ self.origin_input_ids_unpadded = (
+     origin_input_ids_unpadded
+     if origin_input_ids_unpadded is not None
+     else self.origin_input_ids
+ )

After this change, the scheduler proceeds normally and I can continue execution.

Question

Is origin_input_ids expected to always be an array.array before constructing Req?

If so, is there an upstream component in the SMG gRPC path that should already be performing this conversion, or would it make sense for Req.__init__() to normalize the type defensively?

I'd appreciate any guidance on whether this is expected behavior or whether I've missed something in the SMG gRPC integration.

[yudin-s]

Based on current main, Req is now typed as if origin_input_ids is already an array[int], not a plain list[int]. In schedule_batch.py the constructor signature is origin_input_ids: array[int], output_ids is initialized as array("q"), and _refresh_fill_ids() rebuilds with self.origin_input_ids + self.output_ids. So a plain list coming in from the SMG path will hit exactly the failure you saw once decode output is an array.

For the integration, I would treat the gateway/adapter boundary as the first place to fix it: normalize the token ids before constructing Req, so the scheduler keeps one internal representation. That makes the ownership clearer if SMG is bypassing the normal HTTP/tokenizer path.

Your defensive patch in Req.__init__() is still a reasonable upstream hardening patch, but I would make it narrow and explicit:

if not isinstance(origin_input_ids, array):
    origin_input_ids = array("q", origin_input_ids)
if origin_input_ids_unpadded is not None and not isinstance(origin_input_ids_unpadded, array):
    origin_input_ids_unpadded = array("q", origin_input_ids_unpadded)

Also keep the is not None check for origin_input_ids_unpadded; using truthiness changes the behavior for an intentionally empty unpadded input.

So my read is: yes, current SGLang expects origin_input_ids to be array-like by the time Req is built. If the SMG gRPC path can still pass lists, either SMG should convert them before calling into SGLang, or SGLang should accept a small defensive normalization PR to preserve compatibility with older/list-based callers.

If this matches what you are seeing, please mark the answer as accepted so the thread is easier to find later.

[Galmut]

Thanks for the detailed explanation.

That makes sense. Based on your explanation, it seems the SMG gRPC path is still passing list instead of array("q"), while current SGLang expects origin_input_ids to already be an array.

I'll investigate the SMG side to see where the conversion should happen.

I've marked your answer as accepted.

[Galmut]

I added a type check in smg_grpc_servicer/sglang/servicer.py, just before constructing TokenizedGenerateReqInput, and confirmed that input_ids is indeed a Python list at that point.

So it looks like the SMG gRPC servicer is passing a list into SGLang, which later becomes Req.origin_input_ids, while output_ids is now an array("q") in SGLang v0.5.13+.

This seems to match your explanation that the gateway/adapter should normalize the type before constructing Req.