feat(operator): Add MCPServer restart annotation support

[olamide226]

Summary

Adds restart functionality for MCPServer pods via annotations without requiring CRD schema changes. This feature enables users to trigger graceful restarts of MCPServer instances through GitOps workflows and operational commands.

Changes Made

• New Restart Annotations:

  • mcpserver.toolhive.stacklok.dev/restarted-at - RFC3339 timestamp to trigger restart
  • mcpserver.toolhive.stacklok.dev/restart-strategy - Optional strategy selection ("rolling" or "immediate")
  • mcpserver.toolhive.stacklok.dev/last-processed-restart - Internal tracking (set by operator)

• Two Restart Strategies:

  • Rolling (default): Zero-downtime rolling update via deployment pod template annotation
  • Immediate: Fast restart by directly deleting pods for quick recreation

• Non-blocking Error Handling: Invalid annotations or operational failures don't prevent other reconciliation tasks (ToolConfig updates, spec changes, etc.)

• Duplicate Prevention: Tracks last processed restart timestamp to avoid reprocessing same requests

Usage Examples

# Rolling restart (zero downtime, production-safe)
kubectl annotate mcpserver my-server \
  mcpserver.toolhive.stacklok.dev/restarted-at="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --overwrite

# Immediate restart (fast, development/testing)
kubectl annotate mcpserver my-server \
  mcpserver.toolhive.stacklok.dev/restarted-at="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  mcpserver.toolhive.stacklok.dev/restart-strategy="immediate" \
  --overwrite

ref #1880

[Copilot]

Pull Request Overview

This PR adds restart functionality for MCPServer pods through annotations, enabling users to trigger graceful restarts via GitOps workflows without requiring CRD schema changes.

• Implements restart annotation support with two strategies: rolling (zero-downtime) and immediate (fast restart)
• Adds comprehensive test coverage for restart functionality with various scenarios
• Provides documentation and examples for the new restart annotation feature

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
cmd/thv-operator/controllers/mcpserver_controller.go	Core restart functionality implementation with annotation handling and restart strategies
cmd/thv-operator/controllers/mcpserver_restart_test.go	Comprehensive test suite covering restart annotation handling scenarios
docs/operator/restart-annotation.md	Complete documentation for the restart annotation feature
examples/operator/mcp-servers/mcpserver_with_restart_strategy.yaml	Example configuration showing restart annotation usage

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[codecov]

Codecov Report

❌ Patch coverage is 91.07143% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.91%. Comparing base (395b6ba) to head (3db6851).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...d/thv-operator/controllers/mcpserver_controller.go	91.07%	7 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1882      +/-   ##
==========================================
+ Coverage   46.59%   46.91%   +0.31%     
==========================================
  Files         220      220              
  Lines       27201    27319     +118     
==========================================
+ Hits        12674    12816     +142     
+ Misses      13567    13528      -39     
- Partials      960      975      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JAORMX]

The functionality LGTM! Seems you gotta fix the linter warnings and then it's good to go. Thanks a lot for taking a look at this @olamide226 !