diff --git a/.github/ISSUE_TEMPLATE/100-documentation.yml b/.github/ISSUE_TEMPLATE/100-documentation.yml new file mode 100644 index 00000000..f678f9b8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/100-documentation.yml @@ -0,0 +1,22 @@ +name: 📚 Documentation +description: Report an issue related to unifiedcache official website +title: "[Doc]: " +labels: ["documentation"] + +body: +- type: textarea + attributes: + label: 📚 The doc issue + description: > + A clear and concise description of what content in unifiedcache official website is an issue. + validations: + required: true +- type: textarea + attributes: + label: Suggest a potential alternative/fix + description: > + Tell us how we could improve the documentation in this regard. +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/110-user-story.yml b/.github/ISSUE_TEMPLATE/110-user-story.yml new file mode 100644 index 00000000..a662d47c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/110-user-story.yml @@ -0,0 +1,37 @@ +name: 📚 User Story +description: Apply for an user story to be displayed on unifiedcache official website +title: "[User Story]: " +labels: ["user-story"] + +body: +- type: textarea + attributes: + label: 📚 Title + description: > + A clear title about what your user story is about. + validations: + required: true +- type: textarea + attributes: + label: About / Introduction + description: > + A brief introduction about the background of your use case, like your scenario, hardware size etc. +- type: textarea + attributes: + label: Bussiness Challenges + description: > + Tell us how what kind of challenge you faced in this user story. +- type: textarea + attributes: + label: Solving challenges with unifiedcache and benefits + description: > + Tell us how unifiedcache helped you overcome the challenges, including details like how you use it, what version you used, hardware info, etc. And what kind of benefit do you get from using unifiedcache +- type: textarea + attributes: + label: Extra Info + description: > + Any extra infomation you want to include in this story +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/200-installation.yml b/.github/ISSUE_TEMPLATE/200-installation.yml new file mode 100644 index 00000000..c67578fb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/200-installation.yml @@ -0,0 +1,43 @@ +name: 🛠️ Installation +description: Report an issue here when you hit errors during installation. +title: "[Installation]: " +labels: ["installation"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: Your current environment + description: | + Please run the following and paste the output below. + **TODO: Add script to our project to collect the unifiedcache runtime environment, this following example comes from vllm-ascend** + ```sh + npu-smi info + cat /usr/local/Ascend/ascend-toolkit/latest/"$(uname -i)"-linux/ascend_toolkit_install.info + wget https://raw.githubusercontent.com/vllm-project/vllm/main/collect_env.py + # For security purposes, please feel free to check the contents of collect_env.py before running it. + python collect_env.py + ``` + It is suggested to download and execute the latest script, as vllm might frequently update the diagnosis information needed for accurately and quickly responding to issues. + value: | + ```text + The output of `python collect_env.py` + ``` + validations: + required: true +- type: textarea + attributes: + label: How you are installing unifiedcache, also vllm and vllm-ascend. + description: | + Paste the full command you are trying to execute. + value: | + ```sh + pip install -vvv unifiedcache + ``` +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/300-usage.yml b/.github/ISSUE_TEMPLATE/300-usage.yml new file mode 100644 index 00000000..664370cb --- /dev/null +++ b/.github/ISSUE_TEMPLATE/300-usage.yml @@ -0,0 +1,41 @@ +name: 💻 Usage +description: Raise an issue here if you don't know how to use unifiedcache. +title: "[Usage]: " +labels: ["usage"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: Your current environment + description: | + Please run the following and paste the output below. + **TODO: Add script to our project to collect the unifiedcache runtime environment, this following example comes from vllm-ascend** + ```sh + npu-smi info + cat /usr/local/Ascend/ascend-toolkit/latest/"$(uname -i)"-linux/ascend_toolkit_install.info + wget https://raw.githubusercontent.com/vllm-project/vllm/main/collect_env.py + # For security purposes, please feel free to check the contents of collect_env.py before running it. + python collect_env.py + ``` + It is suggested to download and execute the latest script, as vllm might frequently update the diagnosis information needed for accurately and quickly responding to issues. + value: | + ```text + The output of above commands + ``` + validations: + required: true +- type: textarea + attributes: + label: How would you like to use unifiedcache. + description: | + A detailed description of how you want to use unifiedcache. + value: | + I want to run inference of a [specific model](put link here). I don't know how to integrate it with unified. +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/400-bug-report.yml b/.github/ISSUE_TEMPLATE/400-bug-report.yml new file mode 100644 index 00000000..9fc2cf9a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/400-bug-report.yml @@ -0,0 +1,60 @@ +name: 🐛 Bug report +description: Raise an issue here if you find a bug. +title: "[Bug]: " +labels: ["bug"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: Your current environment + description: | + Please run the following and paste the output below. + **TODO: Add script to our project to collect the unifiedcache runtime environment, this following example comes from vllm-ascend** + ```sh + wget https://raw.githubusercontent.com/vllm-project/vllm-ascend/main/collect_env.py + # For security purposes, please feel free to check the contents of collect_env.py before running it. + python collect_env.py + ``` + It is suggested to download and execute the latest script, as vllm might frequently update the diagnosis information needed for accurately and quickly responding to issues. + value: | +
+ The output of `python collect_env.py` + + ```text + Your output of above commands here + ``` + +
+ validations: + required: true +- type: textarea + attributes: + label: 🐛 Describe the bug + description: | + Please provide a clear and concise description of what the bug is. + + If relevant, add a minimal example so that we can reproduce the error by running the code. It is very important for the snippet to be as succinct (minimal) as possible, so please take time to trim down any irrelevant code to help us debug efficiently. We are going to copy-paste your code and we expect to get the same result as you did: avoid any external data, and include the relevant imports, etc. + + If the code is too long (hopefully, it isn't), feel free to put it in a public gist and link it in the issue: https://gist.github.com. + + Please also paste or describe the results you observe instead of the expected results. If you observe an error, please paste the error message including the **full** traceback of the exception. It may be relevant to wrap error messages in ```` ```triple quotes blocks``` ````. + + Please set the environment variable `export VLLM_LOGGING_LEVEL=DEBUG` to turn on more logging to help debugging potential issues. + + If you experienced crashes or hangs, it would be helpful to run vllm with `export VLLM_TRACE_FUNCTION=1` . All the function calls in vllm will be recorded. Inspect these log files, and tell which function crashes or hangs. + placeholder: | + A clear and concise description of what the bug is. + + ```python + # Sample code to reproduce the problem + ``` + + ``` + The error message you got, with the full traceback. + ``` + validations: + required: true diff --git a/.github/ISSUE_TEMPLATE/500-feature-request.yml b/.github/ISSUE_TEMPLATE/500-feature-request.yml new file mode 100644 index 00000000..5c7f9cd0 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/500-feature-request.yml @@ -0,0 +1,31 @@ +name: 🚀 Feature request +description: Submit a proposal/request for a new unifiedcache feature +title: "[Feature]: " +labels: ["feature request"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: 🚀 The feature, motivation and pitch + description: > + A clear and concise description of the feature proposal. Please outline the motivation for the proposal. Is your feature request related to a specific problem? e.g., *"I'm working on X and would like Y to be possible"*. If this is related to another GitHub issue, please link here too. + validations: + required: true +- type: textarea + attributes: + label: Alternatives + description: > + A description of any alternative solutions or features you've considered, if any. +- type: textarea + attributes: + label: Additional context + description: > + Add any other context or screenshots about the feature request. +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/600-new-model.yml b/.github/ISSUE_TEMPLATE/600-new-model.yml new file mode 100644 index 00000000..8654ffb3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/600-new-model.yml @@ -0,0 +1,33 @@ +name: 🤗 Support request for new model supported from huggingface/modelscope/modelers on unifiedcache +description: Submit a proposal/request for a new model from huggingface/modelscope/modelers on unifiedcache +title: "[New Model]: " +labels: ["new model"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). + + #### We also highly recommend you read our official website first to know which model already supported. +- type: textarea + attributes: + label: The model to consider. + description: > + A huggingface/modelscope/modelers url, pointing to the model, e.g. https://huggingface.co/openai-community/gpt2 . + validations: + required: true +- type: textarea + attributes: + label: The closest model unifiedcache already supports. + description: > + Here is the list of models already supported by unifiedcache in our official website . Which model is the most similar to the model you want to add support for? +- type: textarea + attributes: + label: What's your difficulty of supporting the model you want? + description: > + For example, any new operators or new architecture? +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/700-performance-discussion.yml b/.github/ISSUE_TEMPLATE/700-performance-discussion.yml new file mode 100644 index 00000000..c9269351 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/700-performance-discussion.yml @@ -0,0 +1,55 @@ +name: ⚡ Discussion on the performance of unifiedcache +description: Submit a proposal/discussion about the performance of unifiedcache +title: "[Performance]: " +labels: ["performance"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: Proposal to improve performance + description: > + How do you plan to improve unifiedcache's performance? + validations: + required: false +- type: textarea + attributes: + label: Report of performance regression + description: > + Please provide detailed description of performance comparison to confirm the regression. You may want to run the benchmark script at ...**TODO: specify our own benchmark method***. + validations: + required: false +- type: textarea + attributes: + label: Misc discussion on performance + description: > + Anything about the performance. + validations: + required: false +- type: textarea + attributes: + label: Your current environment (if you think it is necessary) + description: | + Please run the following and paste the output below. + **TODO: Add script to our project to collect the unifiedcache runtime environment, this following example comes from vllm-ascend** + ```sh + npu-smi info + cat /usr/local/Ascend/ascend-toolkit/latest/"$(uname -i)"-linux/ascend_toolkit_install.info + wget https://raw.githubusercontent.com/vllm-project/vllm/main/collect_env.py + # For security purposes, please feel free to check the contents of collect_env.py before running it. + python collect_env.py + ``` + It is suggested to download and execute the latest script, as vllm might frequently update the diagnosis information needed for accurately and quickly responding to issues. + value: | + ```text + The output of `python collect_env.py` + ``` + validations: + required: false +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/800-others.yml b/.github/ISSUE_TEMPLATE/800-others.yml new file mode 100644 index 00000000..451e9a8f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/800-others.yml @@ -0,0 +1,21 @@ +name: 🎲 Others +description: Submit a discussion as you like. Note that developers are heavily overloaded and we mainly rely on community users to answer these issues. +title: "[Misc]: " +labels: ["misc"] + +body: +- type: markdown + attributes: + value: > + #### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc). +- type: textarea + attributes: + label: Anything you want to discuss about unifiedcache. + description: > + Anything you want to discuss about unifiedcache. + validations: + required: true +- type: markdown + attributes: + value: > + Thanks for contributing 🎉! diff --git a/.github/ISSUE_TEMPLATE/900-release-checklist.yml b/.github/ISSUE_TEMPLATE/900-release-checklist.yml new file mode 100644 index 00000000..167a201e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/900-release-checklist.yml @@ -0,0 +1,100 @@ +name: Release Checklist +description: Generate a release checklist issue when prepare a new release.(Used for release team) +title: "[Release]: Release checklist for v" + +body: +- type: textarea + attributes: + description: > + Brief info for the new release. + label: Release Checklist + value: > + **Release Version**: + + **Release Branch**: + + **Release Date**: + + **Release Manager**: +- type: textarea + attributes: + description: > + Release notes. + label: Prepare Release Note + value: > + - [ ] Create a new issue for release feedback + + - [ ] Write the release note PR. + + - [ ] Update the feedback issue link in docs/source/faqs.md + + - [ ] Add release note to docs/source/user_guide/release_notes.md + + - [ ] Update version info in docs/source/community/versioning_policy.md + + - [ ] Update contributor info in docs/source/community/contributors.md + + - [ ] Update package version in docs/conf.py +- type: textarea + attributes: + description: > + Make sure the code is merged. + label: PR need Merge + value: > + - [ ] PR link1 + + - [ ] PR link2 + + - [ ] ... +- type: textarea + attributes: + description: > + Make sure the new Feature/Function is tested + label: Functional Test + value: > + - [ ] Feature1 + + - [ ] Bug1 + + - [ ] ... +- type: textarea + attributes: + description: > + Make sure the doc is updated. + label: Doc Test + value: > + - [ ] Tutorial is updated. + + - [ ] User Guide is updated. + + - [ ] Developer Guide is updated. +- type: textarea + attributes: + description: > + Make sure the artifacts is ready + label: Prepare Artifacts + value: > + - [ ] Docker image is ready. + + - [ ] Wheel package is ready. +- type: textarea + attributes: + description: > + Start to release. + label: Release Step + value: > + - [ ] Release note PR is merged. + + - [ ] Post the release on GitHub release page. + + - [ ] Generate official doc page on https://app.readthedocs.org/dashboard/ + + - [ ] Wait for the wheel package to be available on https://pypi.org/project/vllm-ascend + + - [ ] Wait for the docker image to be available on https://quay.io/ascend/vllm-ascend + + - [ ] Upload 310p wheel to Github release page + + - [ ] Broadcast the release news (By message, blog , etc) + + - [ ] Close this issue diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..3ba13e0c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1 @@ +blank_issues_enabled: false diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..b89a2f89 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,34 @@ + + +# Prupose + +What this PR does / why we need it? + + +# Modifications + +Does this PR introduce _any_ user-facing change? + + +# Test + +How was this patch tested? + \ No newline at end of file diff --git a/docs/source/getting-started/example/nfs_conn.md b/docs/source/getting-started/example/nfs_conn.md index 95da8f69..923d63f7 100644 --- a/docs/source/getting-started/example/nfs_conn.md +++ b/docs/source/getting-started/example/nfs_conn.md @@ -13,7 +13,7 @@ When the total size of `kvcache` does not exceed the `max_cache_size` configured ## Features -The DRAM connector supports the following functionalities: +The NFS connector supports the following functionalities: - `dump`: Offload KV cache blocks from HBM to SSD or Local Disk. - `load`: Load KV cache blocks from SSD or Local Disk back to HBM.