[model] support Qwen/Qwen3Guard-Gen-0.6B series #6189
Conversation
Summary of ChangesHello @Jintao-Huang, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request integrates the Qwen3Guard-Gen series of models into the system, enhancing its capabilities with models focused on content moderation and safety evaluation. The changes involve updating internal model and template definitions, registering the new models and their associated safety assessment template, and ensuring that the official documentation in both Chinese and English reflects these additions. The primary goal is to provide robust support for safety-oriented language models. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for the Qwen/Qwen3Guard-Gen model series. The changes include updating model constants, registering the new models and their templates, and adding them to the documentation. The implementation looks correct and consistent with the existing framework. I've added one comment to improve the readability of a long string literal.
| QWEN3_GUARD_TEMPLATE = ( | ||
| '<|im_start|>user\n' | ||
| "# Task:\n1) Evaluate whether THE LAST USER's query belongs to safe or unsafe content based on our " | ||
| 'safety policy.\n2) Identify the category of unsafe content found.\n\n<BEGIN SAFETY POLICY>\n' | ||
| "**Safety: Safe**: The USER's query does not contain any prohibited content or content that could " | ||
| "lead to prohibited content.\n**Safety: Unsafe**: The USER's query contains prohibited content.\n" | ||
| "**Safety: Controversial**: The USER's query does not explicitly contain prohibited content, but " | ||
| 'its intent, context, or potential responses could be misinterpreted, misused, or exploited to ' | ||
| 'generate unsafe content under certain conditions.\n<END SAFETY POLICY>\n\n<BEGIN UNSAFE CONTENT ' | ||
| 'CATEGORIES>\nViolent.\nNon-violent Illegal Acts.\nSexual Content or Sexual Acts.\nPII.\nSuicide ' | ||
| '& Self-Harm.\nUnethical Acts.\nPolitically Sensitive Topics.\nCopyright Violation.\nJailbreak.\n' | ||
| '<END UNSAFE CONTENT CATEGORIES>\n\n<BEGIN CONVERSATION>\n\nUSER: {{QUERY}}\n\n<END CONVERSATION>' | ||
| "\n\nProvide your safety assessment for ONLY THE LAST **USER's query** in the above conversation:\n" | ||
| " - The first line must be one of: 'Safety: Safe', 'Safety: Unsafe', 'Safety: Controversial'.\n - " | ||
| "The second line should start with 'Categories:' followed by a list of any unsafe content " | ||
| "categories, separated by commas. If the content is safe, use 'Categories: None'." | ||
| '<|im_end|>\n<|im_start|>assistant\n') |
There was a problem hiding this comment.
The string literal for QWEN3_GUARD_TEMPLATE is hard to read due to its length, the mix of quote styles, and explicit newlines (\n). Using a triple-quoted string would significantly improve readability and maintainability by making the prompt's structure much clearer.
QWEN3_GUARD_TEMPLATE = """<|im_start|>user
# Task:
1) Evaluate whether THE LAST USER's query belongs to safe or unsafe content based on our safety policy.
2) Identify the category of unsafe content found.
<BEGIN SAFETY POLICY>
**Safety: Safe**: The USER's query does not contain any prohibited content or content that could lead to prohibited content.
**Safety: Unsafe**: The USER's query contains prohibited content.
**Safety: Controversial**: The USER's query does not explicitly contain prohibited content, but its intent, context, or potential responses could be misinterpreted, misused, or exploited to generate unsafe content under certain conditions.
<END SAFETY POLICY>
<BEGIN UNSAFE CONTENT CATEGORIES>
Violent.
Non-violent Illegal Acts.
Sexual Content or Sexual Acts.
PII.
Suicide & Self-Harm.
Unethical Acts.
Politically Sensitive Topics.
Copyright Violation.
Jailbreak.
<END UNSAFE CONTENT CATEGORIES>
<BEGIN CONVERSATION>
USER: {{QUERY}}
<END CONVERSATION>
Provide your safety assessment for ONLY THE LAST **USER's query** in the above conversation:
- The first line must be one of: 'Safety: Safe', 'Safety: Unsafe', 'Safety: Controversial'.
- The second line should start with 'Categories:' followed by a list of any unsafe content categories, separated by commas. If the content is safe, use 'Categories: None'.
<|im_end|>
<|im_start|>assistant
"""3 participants
No description provided.