Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

📌 SakuraLLM Roadmap #42

Open
16 of 32 tasks
pipixia244 opened this issue Jan 18, 2024 · 3 comments
Open
16 of 32 tasks

📌 SakuraLLM Roadmap #42

pipixia244 opened this issue Jan 18, 2024 · 3 comments
Assignees
@pipixia244
Labels
enhancement New feature or request model This issue is about Sakura model server This issue is about Sakura Server API

Comments

@pipixia244
Copy link
Collaborator

pipixia244 commented Jan 18, 2024 •
edited
Loading

模型训练

  • 发布v0.9模型
  • 0.10版本模型(14B,7B,1.8B)
    • prompt内字典
    • 提高指令遵循能力
    • 提高上下文能力
    • 继续增加领域内数据
    • 根据收集的偏好数据进行DPO训练(实验性)
    • 增加润色功能(实验性)
  • Qwen1.5-32B模型
  • 中日韩任意双向互译模型(14B,7B,1.8B)
  • Orion模型
  • Qwen2模型
  • MoE模型
    • deepseek-v2
  • Yi-34B模型

模型量化

  • 发布imatrix量化模型

模型评估

  • 人工问卷形式评估各量化模型性能
  • GPT4评估各量化模型性能

Sakura API

  • 将dev_server分支合并到主分支，并将api格式改为openai like api格式。
  • 支持多种后端至v0.9
    • llama.cpp server
    • llama-cpp-python
    • autogptq
    • vllm(同时支持gptq与awq模型)
  • 适配翻译工具
    • LunaTranslator(新API)
    • GalTransl
    • BallonsTranslator
  • 提供Python部署一键包
  • 支持多线程
  • 支持Batch Inference
  • 支持输出logits
@pipixia244 pipixia244 added enhancement New feature or request model This issue is about Sakura model server This issue is about Sakura Server API labels Jan 18, 2024
@pipixia244 pipixia244 self-assigned this Jan 18, 2024
@pipixia244 pipixia244 pinned this issue Jan 18, 2024
@pipixia244 pipixia244 mentioned this issue Feb 4, 2024
Closed
@neko7055
Copy link

neko7055 commented Apr 8, 2024 •
edited
Loading

或許也可以使用在日文語料訓練微調過的模型試試，此 repo 整理了許多日文的模型，其中 rinna/nekomata-14b 是 qwen-14b continual pre-training在日文語料的模型，可能能增強模型性能

@pipixia244
Copy link
Collaborator Author

或許也可以使用在日文語料訓練微調過的模型試試，此 repo 整理了許多日文的模型，其中 rinna/nekomata-14b 是 qwen-14b continual pre-training在日文語料的模型，可能能增強模型性能

这类模型普遍是日英数据微调，中文能力很差/弱化严重，暂时不做考虑。

@fecet
Copy link

fecet commented Jun 8, 2024

是否有计划开源全精度模型，这样可以借助现在的一些开源推理工具进行部署，它们很多宣传速度已经超过了vllm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pipixia244 pipixia244

Labels
enhancement New feature or request model This issue is about Sakura model server This issue is about Sakura Server API
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fecet @pipixia244 @neko7055