Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation issues despite following README instructions. #106

Closed
saurabh-kadian opened this issue Mar 23, 2024 · 3 comments
Closed

Installation issues despite following README instructions. #106

saurabh-kadian opened this issue Mar 23, 2024 · 3 comments

Comments

@saurabh-kadian
Copy link

saurabh-kadian commented Mar 23, 2024 •
edited

Also thanks @saurabh-kadian, indeed we'll be pushing some updates soon in relation to this. It doesn't seem like it's the case but If you're experiencing any different issues, could I ask you to raise a separate issue?

Originally posted by @lucapericlp in #101 (comment)

@lucapericlp, I am actually having a similar issue. Initially I had problems with installing pipx so I installed everything in a virtual environment.

I ran the following command
poetry run python -i fam/llm/fast_inference.py --quantisation_mode int8, getting the same without quantisation_mode flag as well.

I was not able to get the tts.synthesise command running (name tts is not defined) with the following error trace

Traceback (most recent call last):
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_inference.py", line 161, in <module>
    tts = tyro.cli(TTS)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/tyro/_cli.py", line 187, in cli
    output = _cli_impl(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/tyro/_cli.py", line 454, in _cli_impl
    out, consumed_keywords = _calling.call_from_args(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/tyro/_calling.py", line 241, in call_from_args
    return unwrapped_f(*positional_args, **kwargs), consumed_keywords  # type: ignore
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_inference.py", line 86, in __init__
    self.model, self.tokenizer, self.smodel, self.model_size = build_model(
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_inference_utils.py", line 375, in build_model
    y = generate(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_inference_utils.py", line 211, in generate
    next_token = prefill(model, prompt.view(1, -1).repeat(2, 1), spk_emb, input_pos, **sampling_kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 489, in _fn
    return fn(*args, **kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 655, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 383, in _convert_frame_assert
    compiled_product = _compile(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 646, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 244, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 562, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1033, in transform_code_object
    transformations(instructions, code_options)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 151, in _fn
    return fn(*args, **kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 527, in transform
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2128, in run
    super().run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1213, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/lazy.py", line 90, in realize_and_forward
    return getattr(self.realize(), name)(*args, **kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 328, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1252, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars.items)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 294, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 248, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 81, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1213, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 328, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1252, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars.items)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 294, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 248, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 81, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1213, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/nn_module.py", line 328, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1252, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars.items)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 294, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 248, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/functions.py", line 81, in call_function
    return tx.inline_user_function_return(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2261, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2376, in inline_call_
    tracer.run()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 818, in run
    and self.step()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 781, in step
    getattr(self, inst.opname)(inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 470, in wrapper
    return inner_fn(self, inst)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 1264, in CALL_FUNCTION_KW
    self.call_function(fn, args, kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 652, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/torch.py", line 542, in call_function
    tensor_variable = wrap_fx_proxy(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1314, in wrap_fx_proxy
    return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/variables/builder.py", line 1399, in wrap_fx_proxy_cls
    example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1525, in get_fake_value
    raise TorchRuntimeError(str(e)).with_traceback(e.__traceback__) from None
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1486, in get_fake_value
    ret_val = wrap_fake_exception(
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1027, in wrap_fake_exception
    return fn()
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1487, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1592, in run_node
    raise RuntimeError(fn_str + str(e)).with_traceback(e.__traceback__) from e
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 1571, in run_node
    return node.target(*args, **kwargs)
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <built-in function scaled_dot_product_attention>(*(FakeTensor(..., size=(2, 16, s0, 128)), FakeTensor(..., size=(2, 16, 2048, 128), dtype=torch.float16), FakeTensor(..., size=(2, 16, 2048, 128), dtype=torch.float16)), **{'attn_mask': FakeTensor(..., size=(1, 1, s0, 2048), dtype=torch.bool), 'dropout_p': 0.0}):
Expected query, key, and value to have the same dtype, but got query.dtype: float key.dtype: c10::Half and value.dtype: c10::Half instead.

from user code:
   File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_inference_utils.py", line 131, in prefill
    logits = model(x, spk_emb, input_pos)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_model.py", line 160, in forward
    x = layer(x, input_pos, mask)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_model.py", line 179, in forward
    h = x + self.attention(self.attention_norm(x), mask, input_pos)
  File "/home/sk/.cache/pypoetry/virtualenvs/fam-O9XdsmLv-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/media/sk/HDD/metavoice/metavoice-src/fam/llm/fast_model.py", line 222, in forward
    y = F.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

I figured, it might be due to installation issue of xformers and audiocraft, where I was able to resolve for xformers but not audiocraft. I cleared everything, again following instructions to the tee, fixed pipx, fixed poetry installations, ignoring xformers and audiocraft warnings but still getting the same error and same tts not defined. Not sure what is happening here.

Any hints on how to proceed or what to look for? Thanks in advance !
@saurabh-kadian saurabh-kadian mentioned this issue Mar 23, 2024
Closed
@saurabh-kadian
Copy link
Author

saurabh-kadian commented Mar 23, 2024 •
edited

Closing this issue.

I was able to move ahead by making the query from type float to type half, by adding the line q = q.half() on line 221 in fam/llm/fast_model.py. Not sure how this would impact the performance, as of now.

I am able to use tts object from fast_inference.py in the interactive session now.

Thanks !!

@lucapericlp
Copy link
Contributor

Thanks for raising, resolving & letting us know @saurabh-kadian! cc @vatsalaggarwal, have we bumped into this before?

@lucapericlp lucapericlp mentioned this issue Mar 24, 2024
Open
@vatsalaggarwal
Copy link
Member

Let's track in #108 ... we had bumped into as we don't do run stuff much with fp16, and had missed it -- but I thought @sidroopdaska fixed it

@lucapericlp lucapericlp mentioned this issue May 14, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vatsalaggarwal @saurabh-kadian @lucapericlp