TypeError: forward() got an unexpected keyword argument 'position_ids'

[ScientiaEtVeritas]

I started a server with

text-generation-launcher --model-id facebook/galactica-30b --num-shard 1

However, when I now send a request like

curl localhost:3000/generate -H 'Content-Type: application/json' -d '{"inputs":"Hi my name is","parameters":{"max_new_tokens":60}}'

it consistenly returns TypeError: forward() got an unexpected keyword argument 'position_ids', with the following traceback:

2023-02-20T14:41:54.081514Z ERROR shard-manager: text_generation_launcher: "Method Prefill encountered an error.
Traceback (most recent call last):
  File \"/home/user/miniconda3/envs/text_generation/bin/text-generation-server\", line 8, in <module>
    sys.exit(app())
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/typer/main.py\", line 311, in __call__
    return get_command(self)(*args, **kwargs)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/click/core.py\", line 1130, in __call__
    return self.main(*args, **kwargs)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/typer/core.py\", line 778, in main
    return _main(
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/typer/core.py\", line 216, in _main
    rv = self.invoke(ctx)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/click/core.py\", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/click/core.py\", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/click/core.py\", line 760, in invoke
    return __callback(*args, **kwargs)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/typer/main.py\", line 683, in wrapper
    return callback(**use_params)  # type: ignore
  File \"/home/user/text-generation-inference-new/server/text_generation/cli.py\", line 55, in serve
    server.serve(model_id, revision, sharded, quantize, uds_path)
  File \"/home/user/text-generation-inference-new/server/text_generation/server.py\", line 130, in serve
    asyncio.run(serve_inner(model_id, revision, sharded, quantize))
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/asyncio/runners.py\", line 44, in run
    return loop.run_until_complete(main)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/asyncio/base_events.py\", line 634, in run_until_complete
    self.run_forever()
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/asyncio/base_events.py\", line 601, in run_forever
    self._run_once()
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/asyncio/base_events.py\", line 1905, in _run_once
    handle._run()
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/asyncio/events.py\", line 80, in _run
    self._context.run(self._callback, *self._args)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/grpc_interceptor/server.py\", line 153, in invoke_intercept_method
    return await self.intercept(
> File \"/home/user/text-generation-inference-new/server/text_generation/interceptor.py\", line 20, in intercept
    return await response
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py\", line 82, in _unary_interceptor
    raise error
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py\", line 73, in _unary_interceptor
    return await behavior(request_or_iterator, context)
  File \"/home/user/text-generation-inference-new/server/text_generation/server.py\", line 41, in Prefill
    generations, next_batch = self.model.generate_token(batch)
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/contextlib.py\", line 79, in inner
    return func(*args, **kwds)
  File \"/home/user/text-generation-inference-new/server/text_generation/models/causal_lm.py\", line 297, in generate_token
    logits, past = self.forward(
  File \"/home/user/text-generation-inference-new/server/text_generation/models/causal_lm.py\", line 284, in forward
    outputs = self.model.forward(
  File \"/home/user/miniconda3/envs/text_generation/lib/python3.9/site-packages/accelerate/hooks.py\", line 156, in new_forward
    output = old_forward(*args, **kwargs)
TypeError: forward() got an unexpected keyword argument 'position_ids'
" rank=0
2023-02-20T14:41:54.081879Z ERROR batch{batch_size=1}:prefill:prefill{id=0 size=1}:prefill{id=0 size=1}: text_generation_client: router/client/src/lib.rs:29: Server error: forward() got an unexpected keyword argument 'position_ids'
2023-02-20T14:41:54.081947Z ERROR HTTP request{otel.name=POST /generate http.client_ip= http.flavor=1.1 http.host=localhost:3000 http.method=POST http.route=/generate http.scheme=HTTP http.target=/generate http.user_agent=curl/7.82.0 otel.kind=server trace_id=c742e54c2eddc1bfcc788ed10b3e0c52}:generate{req=Json(GenerateRequest { inputs: "Hi my name is", parameters: GenerateParameters { temperature: None, repetition_penalty: None, top_k: None, top_p: None, do_sample: false, max_new_tokens: 60, stop: [], details: false, seed: None } })}:generate{request=GenerateRequest { inputs: "Hi my name is", parameters: GenerateParameters { temperature: None, repetition_penalty: None, top_k: None, top_p: None, do_sample: false, max_new_tokens: 60, stop: [], details: false, seed: None } }}:generate_stream{request=GenerateRequest { inputs: "Hi my name is", parameters: GenerateParameters { temperature: None, repetition_penalty: None, top_k: None, top_p: None, do_sample: false, max_new_tokens: 60, stop: [], details: false, seed: None } }}:infer{batch_size=1}:send_error: text_generation_router::infer: router/src/infer.rs:338: Request failed during generation: Server error: forward() got an unexpected keyword argument 'position_ids'

[OlivierDehaene]

fixed on main.