'activation_ops' circular import

[sigmareaver]

Attempted to run the example code in the README.md.
Model: Metharme-13b-GPTQ

Traceback (most recent call last):
  File "/mnt/Storage/ai-dev/aphrodite-engine/inference_test.py", line 1, in <module>
    from aphrodite import LLM, SamplingParams
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/__init__.py", line 2, in <module>
    from aphrodite.engine.async_aphrodite import AsyncAphrodite
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/engine/async_aphrodite.py", line 7, in <module>
    from aphrodite.engine.aphrodite_engine import AphroditeEngine
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/engine/aphrodite_engine.py", line 14, in <module>
    from aphrodite.task_handler.worker import Worker
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/task_handler/worker.py", line 5, in <module>
    from aphrodite.modeling import get_model, InputMetadata, set_random_seed
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/modeling/__init__.py", line 2, in <module>
    from aphrodite.modeling.loader import get_model
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/modeling/loader.py", line 7, in <module>
    from aphrodite.modeling.models import LlamaForCausalLM, GPTJForCausalLM, GPTNeoXForCausalLM
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/modeling/models/__init__.py", line 1, in <module>
    from aphrodite.modeling.models.llama import LlamaForCausalLM
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/modeling/models/llama.py", line 33, in <module>
    from aphrodite.modeling.layers.activation import SiluAndMul
  File "/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/modeling/layers/activation.py", line 4, in <module>
    from aphrodite import activation_ops
ImportError: cannot import name 'activation_ops' from partially initialized module 'aphrodite' (most likely due to a circular import) (/mnt/Storage/ai-dev/aphrodite-engine/aphrodite/__init__.py)

[AlpinDale]

Did you forget to install Aphrodite with pip install -e .? Another thing to note is that GPTQ models are not supported - only 16-bit HuggingFace weights. I'll update the README to explicitly men5ion that.

[sigmareaver]

Ah, okay.

Retested with RedPajama-INCITE-Base-3B-v1 but the error persists.

[sigmareaver]

User error. Did pip install e . instead of pip install -e .