Skip to content

feat: SHS MCP Optimizaiton #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Jun 28, 2025
Merged

feat: SHS MCP Optimizaiton #1

merged 13 commits into from
Jun 28, 2025
+5,965 −170

Conversation

@vara-bonthu
Copy link
Collaborator

@vara-bonthu vara-bonthu commented Jun 28, 2025 •
edited
Loading

Key changes in this PR

  • Core project structure and licensing - Apache License, pyproject.toml, Makefile, .gitignore
  • Security and configuration fixes - Environment variables, authentication, error handling
  • Enhanced documentation - README.md, TESTING.md, CONTRIBUTING.md, user guides
  • Test coverage additions - 19 new tests increasing coverage from 18% to 47%
  • Integration examples and deployment - AI framework integrations, Kubernetes Helm charts
  • CI/CD and development workflow - GitHub Actions, pre-commit hooks, Docker improvements

Testing

  • Performed local testing using TESTING.md file

Output

image

Testing evidence

vara@ spark-history-server-mcp % pre-commit run -a
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Passed
check json...........................................(no files to check)Skipped
check toml...............................................................Passed
check for merge conflicts................................................Passed
check for added large files..............................................Passed
mixed line ending........................................................Passed
ruff check...............................................................Passed
ruff format..............................................................Passed
bandit...................................................................Passed
markdownlint.............................................................Passed

vara-bonthu added 11 commits June 27, 2025 16:55
@vara-bonthu
feat: establish core project structure with Apache License
94c9260 
- Add Apache License 2.0 with authors Manabu McCloskey and Vara Bonthu
- Update pyproject.toml with proper metadata, keywords, and classifiers
- Add comprehensive Makefile for development workflows
- Enhance .gitignore with Spark, MCP, and platform-specific exclusions
@vara-bonthu
fix: implement critical security and configuration improvements
3ab6773 
- Fix debug mode to use environment variable instead of hardcoded true
- Add environment variable support for authentication in config.py
- Remove hardcoded credentials from config.yaml
- Add proper error handling and logging in main.py
- Fix typo "sever_config" to "server_config" in spark clients
- Add .env.example for secure environment variable setup
@vara-bonthu
docs: create comprehensive documentation and user guides
759f763 
- Rewrite README.md with emoji-rich engaging content for open source
- Add detailed TESTING.md with MCP Inspector and local testing guides
- Create CONTRIBUTING.md with setup instructions and guidelines
- Add .markdownlint.yaml for consistent documentation formatting
- Update examples/basic/README.md with clear setup instructions
- Add start_local_spark_history.sh script for easy local testing
- Create screenshots/ directory for future MCP Inspector images
@vara-bonthu
test: add comprehensive test coverage for MCP tools
dae74f0 
- Add 19 new tests for critical MCP tools in test_tools.py
- Test coverage increased from 18% to 47% with tests for:
  - get_application: success, not found, and error scenarios
  - get_jobs: success, empty results, and error handling
  - get_stages: success, invalid app ID, and error scenarios
  - get_stage_task_summary: success, not found, and error cases
  - get_slowest_sql_queries: success, no queries, and error handling
- Enhance tools.py with improved error handling and validation
- Add proper mock testing for Spark client interactions
@vara-bonthu
feat: add AI framework integrations and Kubernetes deployment
680a6a4 
- Create examples/integrations/ with comprehensive AI framework guides:
  - Claude Desktop integration with troubleshooting
  - LangChain integration for RAG applications
  - AutoGen multi-agent workflow integration
  - Custom AI agent implementation examples
- Add complete Kubernetes deployment with Helm charts:
  - Production-ready Helm templates with security best practices
  - ConfigMap, Secret, Service, and Deployment manifests
  - Horizontal Pod Autoscaling and monitoring setup
  - values.yaml with comprehensive configuration options
- Include sample Spark event logs for testing in examples/basic/events/
@vara-bonthu
ci: establish comprehensive CI/CD and development workflows
b1c1b76 
- Add GitHub Actions CI workflow with Python 3.12/3.13 matrix testing
- Configure automated testing, linting, and security checks
- Add pull request template with testing checklist and guidelines
- Enhance .pre-commit-config.yaml with community-ready hooks
- Update Dockerfile with security best practices and multi-stage build
- Update uv.lock with latest dependency resolution
@vara-bonthu
docs: add work-in-progress note for Kubernetes deployment
a350d55 
- Add warning note about container image and Helm registry publishing
- Clarify that deployment setup is still being tested
@vara-bonthu
docs: add work-in-progress note and MCP Inspector screenshots
3354619 
- Add warning note about container image and Helm registry publishing
- Include MCP Inspector screenshots for get-application and job-compare tools
- Update uv.lock with latest dependencies
@vara-bonthu
fix: add missing Code Quality Checks job and pre-commit dependency
e1a821c 
- Add Code Quality Checks job to CI workflow with pre-commit installation
- Include pre-commit>=3.0.0 in dev dependencies
- Fix CI failure by ensuring pre-commit is available in build environment
@vara-bonthu
fix: resolve pre-commit security and linting issues
86d75ae 
- Add timeout=30 to requests.get calls to fix S113 security warning
- Use tempfile.gettempdir() instead of hardcoded /tmp for S108 security warning
- Update bandit args to use long form for compatibility
- Exclude Helm templates from YAML validation (they contain Go templates)
@vara-bonthu
fix: apply pre-commit formatting and temporarily disable mypy
1b752c9 
- Apply ruff formatting fixes to ensure consistent code style
- Temporarily disable mypy type checking in pre-commit and CI
- Add TODO to re-enable mypy after fixing type annotations
- All security and linting checks now pass
@vara-bonthu vara-bonthu requested a review from nabuskey June 28, 2025 02:18
@vara-bonthu
fix: update tests to match security improvements
7925e7f 
- Add timeout=30 parameter to mock requests.get calls
- Fix get_jobs test to expect status=None parameter
- Fix filtering tests to mock client returning filtered results
- Fix SQL queries test to specify top_n=2 for multiple results
- All tests now align with actual function implementations
@vara-bonthu
fix: correct CI badge URL to show main branch status
3dde084 
- Update CI badge to use ?branch=main parameter
- Badge will show proper status once PR is merged to main branch
nabuskey
nabuskey approved these changes Jun 28, 2025
View reviewed changes
Copy link
Collaborator

@nabuskey nabuskey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Opened a couple of issues based on some questions I've had.

@nabuskey nabuskey merged commit 2b757ca into main Jun 28, 2025
4 checks passed
@vara-bonthu vara-bonthu deleted the shs-mcp-optimization branch July 24, 2025 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nabuskey nabuskey nabuskey approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vara-bonthu @nabuskey