Discussions

• 1 You must be logged in to vote

  🙏

  quantize.py vs convert_checkpoint.py

  DreamGenX asked May 15, 2024 in Q&A · Unanswered

  

  

  0
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced May 14, 2024 in Announcements

  

  

  0
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced May 7, 2024 in Announcements

  

  

  0
• 7 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Apr 30, 2024 in Announcements

  

  

  0
• 3 You must be logged in to vote

  🙏

  With the new TensorRT-LLM 0.9.0, ModelConfig in tesnorrt_llm.runtime.generation now has new args max_batch_size, max_beam_width - how do we set these?

  digitalmonkey asked Apr 26, 2024 in Q&A · Unanswered

  

  

  0
• 3 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Apr 16, 2024 in Announcements

  

  

  

  

  4
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Apr 24, 2024 in Announcements

  

  

  0
• 1 You must be logged in to vote

  🙏

  TRT/TRT-LLM support for NestedTensor / jagged or ragged tensors for doing away with padding

  vadimkantorov asked Apr 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  How do I use idefics2-8b with TensorRT LLM?

  Iven2132 asked Apr 17, 2024 in Q&A · Unanswered

  

  

  0
• 9 You must be logged in to vote

  📣

  TensorRT-LLM 0.9.0 Release

  kaiyux announced Apr 16, 2024 in Announcements

  

  

  0
• 4 You must be logged in to vote

  💬

  Can we please have meaningful commit message in the spirit of open source

  ekagra-ranjan started Apr 13, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Feb 21, 2024 in Announcements

  

  

  

  

  12
• 2 You must be logged in to vote

  💬

  why is the prompt_table in ModelRunner.generate passed in as npy file instead of a tensor ?

  yupbank started Apr 9, 2024 in General

  

  

  0
• 8 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Apr 9, 2024 in Announcements

  

  

  0
• 5 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Apr 1, 2024 in Announcements

  

  

  0
• 4 You must be logged in to vote

  📣

  Update on the development branch

  Shixiaowei02 announced Mar 26, 2024 in Announcements

  

  

  0
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Mar 19, 2024 in Announcements

  

  

  1
• 1 You must be logged in to vote

  🙏

  Where can I find trtllm-build?

  hossainiir asked Mar 17, 2024 in Q&A · Closed · Unanswered

  

  

  1
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Feb 27, 2024 in Announcements

  

  

  1
• 4 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Mar 12, 2024 in Announcements

  

  

  0
• 1 You must be logged in to vote

  🙏

  When to support Yi model

  hanmo-1 asked Mar 11, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Mar 5, 2024 in Announcements

  

  

  0
• 4 You must be logged in to vote

  📣

  TensorRT-LLM 0.8.0 Release

  kaiyux announced Feb 29, 2024 in Announcements

  

  

  0
• 1 You must be logged in to vote

  🙏

  My llava1.5 7b can get output, but how to set it up to get the same output as transformers

  bleedingfight asked Feb 28, 2024 in Q&A · Unanswered

  

  

  0
• 8 You must be logged in to vote

  📣

  Update on the development branch

  kaiyux announced Feb 6, 2024 in Announcements

  

  

  

  

  2