[bugfix] patch deepseek runtime

[slin1237]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

[gemini-code-assist]

Summary of Changes

Hello @slin1237, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug in the Deepseek runtime configuration by updating the YAML definition for the SGLang server deployment. The changes focus on refining container settings, optimizing server launch parameters, and adjusting network and resource configurations to improve the stability and performance of the Deepseek runtime environment.

Highlights

• Runtime Configuration Updates: Updated the deepseek-rdma-pd-rt.yaml configuration to specify GreaterThanOrEqual operators for modelFormat and modelFramework versions, providing more flexible version compatibility.
• Container Image and Environment Changes: The SGLang container image has been updated from v0.4.8.post1-cu126 to dev. Several environment variables and command-line arguments for the SGLang server launch have been modified or removed to align with the new image and optimize runtime behavior, including changes to port numbers and IB device specification.
• Kubernetes Pod Configuration Enhancements: Added enableServiceLinks: false, hostIPC: true, and mounted /dev/shm and /dev/infiniband volumes to the leader and worker pods. Security contexts were also added to grant IPC_LOCK and CAP_SYS_ADMIN capabilities and set privileged: true for the containers, which are crucial for RDMA and shared memory operations.
• Router Policy and Parameters Adjustment: The router's load balancing policy was changed from power_of_two to random. The service-discovery-port was updated to 30000, and new parameters --max-payload-size and --worker-startup-timeout-secs were introduced to the router configuration.
• Health Probe Removal: Removed explicit readinessProbe, livenessProbe, and startupProbe configurations from the engineConfig and decoderConfig leader runners.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates the DeepSeek runtime configuration, primarily by cleaning up supported model formats, restructuring the commands, and adding configurations for RDMA. While these changes seem to improve the configuration's structure, there are several critical issues introduced. All health probes have been removed, which will severely impact service reliability. Additionally, containers are now running with privileged: true, which is a major security concern. The configuration file also suffers from extensive duplication, making it hard to maintain. Finally, the use of the :dev image tag makes deployments non-reproducible. My review provides specific feedback on these points with suggestions for improvement.