Refactor codebase with modular architecture and multi-kernel support #2
Conversation
- Modularized Jupyter client logic into separate jupyter_client.py - Created centralized configuration system with environment variable support - Improved error handling with custom exception classes - Optimized Dockerfile with multi-stage build and tini for signal handling - Enhanced entrypoint.sh with proper error handling and cleanup - Added comprehensive test suite with pytest - Pinned all dependencies to specific versions for reproducible builds - Updated README.md with development guide and configuration options
- Enhanced JupyterClient to support both Python and Bash kernels - Added execute_bash_code MCP tool for shell command execution - Added get_kernel_status tool to monitor kernel availability - Updated entrypoint.sh to start both Python and Bash kernels - Extended test suite with comprehensive bash kernel tests - Updated documentation with usage examples and tool descriptions - Maintained backward compatibility with existing Python-only usage
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @abhishek-anand, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request represents a significant architectural overhaul of CodeRunner, moving towards a more robust, modular, and extensible system. The primary goal is to enhance the platform's capabilities by introducing multi-language support (Python and Bash) for code execution, alongside improvements in configuration management, container efficiency, and overall developer experience. This refactoring lays a solid foundation for future feature development while maintaining backward compatibility.
Highlights
- Modular Architecture: The codebase has been significantly refactored into a more modular architecture, extracting Jupyter client logic into a dedicated
jupyter_client.pymodule and centralizing configuration in a newconfig.pyfile. This improves separation of concerns and maintainability. - Multi-Kernel Support: CodeRunner now supports multiple Jupyter kernels, specifically adding a Bash kernel alongside the existing Python kernel. This enables execution of both Python code and Bash scripts, expanding the capabilities for AI models.
- Enhanced Configuration: A new centralized configuration system has been implemented using environment variables (prefixed with
CODERUNNER_) and sensible defaults, improving flexibility for deployment and local development. - New MCP Tools: Two new MCP tools have been introduced:
execute_bash_codefor running shell commands and scripts, andget_kernel_statusto monitor the availability of the different Jupyter kernels. - Container Optimizations: The Docker build process has been optimized with a multi-stage build, resulting in smaller image sizes. Proper signal handling has been integrated using
tini, and theentrypoint.shscript is enhanced for better error handling and automatic kernel startup. - Improved Testing: A comprehensive test suite using
pytesthas been added, covering configuration and Jupyter client functionality with mock-based testing for isolated components.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Summary of Changes
Hello @abhishek-anand, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request delivers a substantial architectural overhaul, transforming the codebase into a more modular and robust system. The primary goal is to enhance flexibility by introducing multi-kernel support for both Python and Bash execution, alongside significant improvements in configuration management, container efficiency, and overall developer experience. These changes aim to make the application more scalable, easier to maintain, and more versatile for AI-driven code execution.
Highlights
- Modular Architecture: The codebase has been significantly refactored into a more modular structure. Key components like Jupyter client communication are now encapsulated in
jupyter_client.py, and configuration management is centralized in a newconfig.pymodule, improving separation of concerns and maintainability. - Multi-Kernel Support: The system now supports multiple Jupyter kernels, specifically Python and Bash. This enables the execution of both Python code and shell commands, with new MCP tools (
execute_bash_code) introduced to leverage this capability. Kernel IDs are automatically managed and loaded. - Enhanced Configuration: A new centralized configuration system has been implemented, allowing settings to be managed via environment variables (prefixed with
CODERUNNER_). This provides greater flexibility for deployment and local development, with a new.env.examplefile for guidance. - Container Optimizations: The Docker build process has been optimized using a multi-stage approach, resulting in smaller image sizes. The
entrypoint.shscript has been enhanced for better error handling and proper signal management usingtini, ensuring graceful shutdowns and reliable kernel startup. - Improved Developer Experience & Testing: Dependencies are now pinned for reproducible builds. A comprehensive test suite using
pytesthas been added, covering configuration and Jupyter client functionality, including mock-based testing. Documentation has also been updated to reflect these changes and provide usage examples.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The pull request introduces a modular architecture with multi-kernel support for both Python and Bash execution. The changes include new files for configuration and Jupyter client, modifications to the Dockerfile and entrypoint script, and updates to the README. The code has been refactored to improve maintainability and add new features such as bash code execution and kernel status monitoring. I have provided feedback to improve the Dockerfile, entrypoint script, and jupyter_client.
| while ! curl -s --fail "http://localhost:${JUPYTER_PORT}/api/status" > /dev/null; do | ||
| count=$((count + 1)) | ||
|
|
||
| if [ "$count" -gt "$MAX_WAIT" ]; then | ||
| echo "Error: Jupyter Server did not start within ${MAX_WAIT} seconds." | ||
| kill $JUPYTER_PID 2>/dev/null || true | ||
| exit 1 |
There was a problem hiding this comment.
It's good to include kill $JUPYTER_PID 2>/dev/null || true before exiting to ensure that the Jupyter server process is terminated if it fails to start within the specified timeout. This prevents orphaned processes and ensures that resources are released properly. The 2>/dev/null || true part is added to ignore errors if the process doesn't exist.
| while ! curl -s --fail "http://localhost:${JUPYTER_PORT}/api/status" > /dev/null; do | |
| count=$((count + 1)) | |
| if [ "$count" -gt "$MAX_WAIT" ]; then | |
| echo "Error: Jupyter Server did not start within ${MAX_WAIT} seconds." | |
| kill $JUPYTER_PID 2>/dev/null || true | |
| exit 1 | |
| echo "Error: Jupyter Server did not start within ${MAX_WAIT} seconds." | |
| kill $JUPYTER_PID 2>/dev/null || true | |
| exit 1 |
| if [ $? -ne 0 ]; then | ||
| echo "Warning: Failed to start Bash kernel (continuing without it)" |
There was a problem hiding this comment.
Consider exiting with a non-zero exit code if the bash kernel fails to start, as this indicates a failure in setting up the environment. This can be achieved by adding an exit 1 command after the echo command.
| if [ $? -ne 0 ]; then | |
| echo "Warning: Failed to start Bash kernel (continuing without it)" | |
| echo "Warning: Failed to start Bash kernel (continuing without it)" | |
| exit 1 |
| if kernel_type not in self.kernels: | ||
| raise JupyterConnectionError(f"Unsupported kernel type: {kernel_type}") |
There was a problem hiding this comment.
Raising a ValueError might be more appropriate than JupyterConnectionError for an unsupported kernel type, as it indicates an invalid argument was passed to the function.
raise ValueError(f"Unsupported kernel type: {kernel_type}")| except Exception as e: | ||
| logger.error(f"Unexpected error during execution: {e}", exc_info=True) | ||
| raise JupyterExecutionError(f"Internal error during execution: {str(e)}") |
There was a problem hiding this comment.
It's better to re-raise the exception using raise instead of creating a new JupyterExecutionError with the string representation of the original exception. This preserves the original traceback and exception type, which can be helpful for debugging.
raise
There was a problem hiding this comment.
Code Review
This pull request introduces a modular architecture with multi-kernel support, enhancing the project's maintainability and functionality. The changes include a dedicated JupyterClient and centralized configuration. The review suggests aligning environment variable usage in the entrypoint script and improving the robustness of kernel file discovery.
- Updated entrypoint.sh to use CODERUNNER_ prefix for all environment variables - Ensures consistency across Python config, Docker environment, and shell script - Provides unified configuration surface for the entire application - Updated documentation to highlight this consistency improvement
- Replace brittle string replacement with robust path construction using os.path.dirname() - Add bash_kernel_id_file property to Config class for centralized path management - Update tests to use new robust approach - Add test coverage for custom kernel file path scenarios - Ensures compatibility with any custom CODERUNNER_KERNEL_ID_FILE configuration
- Add test for file read error handling in kernel ID loading - Add test for resource settings with CPU conversion (100% config.py coverage) - Add test for execution timeout scenarios - Fix default directory path test for cross-environment compatibility - Comprehensive coverage: config.py (100%), jupyter_client.py (95%) - Total 23 passing tests with robust error condition coverage
Summary
This PR introduces a major refactoring of the codebase with improved architecture, better error handling, and multi-kernel support for both Python and Bash execution.
Key Changes
🏗️ Architecture Improvements
🔧 New Features
🐳 Container Optimizations
🛠️ Developer Experience
🧪 Testing & Quality
Migration Guide
For Users
No changes required - existing usage continues to work. New bash execution capability is available via execute_bash_code tool.
For Developers
Testing
File Changes
Breaking Changes
None - this is a backward-compatible refactoring.