Skip to content

issues Search Results · repo:abacusai/gh200-llm language:Python

Filter by

11 results
 (48 ms)

11 results

inabacusai/gh200-llm (press backspace or delete to remove)

abacusai/gh200-llm
ADD LMCache to the docker build

vLLM now uses LMCache for KV caching and offloading. Can you include that into the docker build?
  • khayamgondal
  • Opened 
    3 days ago
  • #12

abacusai/gh200-llm
ImportError: /usr/local/lib/python3.12/dist-packages/torch/lib/libc10_cuda.so: undefined symbol

I m running it using apptainer. $ apptainer pull docker://ghcr.io/abacusai/gh200-llm/llm-train-serve:latest $ apptainer run apptainer run --nv --bind /projects llm-train-serve_latest.sif /bin/bash --login ...
  • zkx06111
  • 1
  • Opened 
    25 days ago
  • #11

abacusai/gh200-llm
Contribute vLLM wheel build or dockerfile to upstream?

Hello and thank you for your great image! I am a committer on the vLLM project and wanted to ask if you would be interested in contributing an official gh200 dockerfile to vLLM. The current dockerfile ...
  • mgoin
  • Opened 
    on Jan 29
  • #10

abacusai/gh200-llm
Docker image for Cuda 12.4

Hi I noticed that the latest release image is for Cuda 12.6. Do you also have an image for an older version where Cuda 12.4 is supported? Best Manthan
  • manthan99
  • Opened 
    on Jan 17
  • #9

abacusai/gh200-llm
Would it possible to upgrade vllm to 0.6.3 to support llama3.2

I tried but failed, could you please update?
  • alexchenyu
  • Opened 
    on Oct 17, 2024
  • #8

abacusai/gh200-llm
Train script error: No module named 'data.finetune_data'

Seeing following error when running train.sh data==0.4 installed Using your latest prebuilt docker image Launching script via bash training/train.sh [2024-09-26 18:05:30,229] [INFO] [launch.py:256:main] ...
  • khayamgondal
  • Opened 
    on Sep 26, 2024
  • #7

abacusai/gh200-llm
[Feature Request] Provide test scripts

Your Numbers are looking Promising. We experimented with the GH200 and vLLM too, but could not reproduce your high numbers, so we are highly interested in the acutal scripts which produced this numbers. ...
  • Wetzr
  • Opened 
    on Sep 5, 2024
  • #5

abacusai/gh200-llm
[Feature-Request] Provide build scripts and docker containers for .whl files

In the docker file you have references to custom built .whl files (e.g. https://static.abacus.ai/pypi/abacusai/gh200-llm/pytorch-2407-cuda125/flash_attn-2.6.3-cp310-cp310-linux_aarch64.whl) but there are ...
  • agm-eratosth
  • Opened 
    on Aug 28, 2024
  • #4

abacusai/gh200-llm
[Feature Request] - Liger Kernels on ARM64 for GH200?

Hi, @arvindsun I m not sure if you ve seen this repository from the folks at LinkedIn, but I was wondering on the off chance you did if you d had any luck getting this to work on GH200? I ve tried adding ...
  • jlotthammer
  • 2
  • Opened 
    on Aug 24, 2024
  • #3

abacusai/gh200-llm
Update VLLM

The VLLM version used in dockerfile is 4.1 The docker container seems to use precompiled pypi packages. Please provide an update for VLLM version 5.0 or insight into how to update this to 5.0
  • NiklasWilson
  • 1
  • Opened 
    on Jun 12, 2024
  • #2
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Restrict your search to the title by using the in:title qualifier.
Issue origami icon

Learn how you can use GitHub Issues to plan and track your work.

Save views for sprints, backlogs, teams, or releases. Rank, sort, and filter issues to suit the occasion. The possibilities are endless.Learn more about GitHub Issues
ProTip! 
Press the
/
key to activate the search input again and adjust your query.
Issue search results · GitHub