We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Check upon issue creation:
For XXX in:
Parameters:
NEXT_MODEL_PATH=microsoft/Phi-3-mini-XXX-instruct NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=4
ToDos:
The text was updated successfully, but these errors were encountered:
Seems to be an issue with VLLM with this model:
2024-06-09T01:10:42.575996000Z 2024-06-09 01:10:42,575 - root - INFO - Formatted MC-Question-Block for lsat-lr dataset 2024-06-09T01:10:42.576031126Z 2024-06-09 01:10:42,575 - root - INFO - Loading vLLM model microsoft/Phi-3-mini-128k-instruct 2024-06-09T01:10:44.403415347Z Traceback (most recent call last): 2024-06-09T01:10:44.403433791Z File "/usr/local/bin/cot-eval", line 8, in <module> 2024-06-09T01:10:44.403459111Z sys.exit(main()) 2024-06-09T01:10:44.403472573Z File "/workspace/cot-eval/src/cot_eval/__main__.py", line 149, in main 2024-06-09T01:10:44.403507563Z llm = VLLM( 2024-06-09T01:10:44.403513620Z File "/usr/local/lib/python3.10/dist-packages/langchain_core/load/serializable.py", line 120, in __init__ 2024-06-09T01:10:44.403545765Z super().__init__(**kwargs) 2024-06-09T01:10:44.403551285Z File "/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py", line 341, in __init__ 2024-06-09T01:10:44.403604111Z raise validation_error 2024-06-09T01:10:44.403631526Z pydantic.v1.error_wrappers.ValidationError: 1 validation error for VLLM 2024-06-09T01:10:44.403633889Z __root__ 2024-06-09T01:10:44.403635718Z (type=assertion_error)
Sorry, something went wrong.
yakazimir
No branches or pull requests
Check upon issue creation:
For XXX in:
Parameters:
ToDos:
The text was updated successfully, but these errors were encountered: