feat: Add per-pool scheduling and fix critical RustFS compatibility bugs #38
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds advanced multi-pool scheduling capabilities and fixes
critical bugs discovered through comprehensive analysis of the RustFS
source code (~/git/rustfs).
## Critical Bug Fixes
Verified against RustFS source code (crates/config/src/constants/app.rs):
1. Fix console port: 9090 → 9001
- RustFS DEFAULT_CONSOLE_ADDRESS is ":9001", not 9090
- Affects: services.rs, workloads.rs
2. Fix IO service port: 90 → 9000
- S3 API standard port is 9000
- RustFS DEFAULT_ADDRESS is ":9000"
- Affects: services.rs
3. Add required RustFS environment variables:
- RUSTFS_ADDRESS=0.0.0.0:9000
- RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
- RUSTFS_CONSOLE_ENABLE=true
- Without these, RustFS containers fail to start properly
- Verified from RustFS docker-compose.yml and Helm chart
4. Standardize volume paths to RustFS convention:
- Before: /data/{N} (custom)
- After: /data/rustfs{N} (RustFS standard)
- RUSTFS_VOLUMES: .../data/rustfs{0...N}
- Matches RustFS Helm chart, MNMD examples, docker-compose
## Multi-Pool Scheduling Enhancements
Added comprehensive Kubernetes scheduling capabilities per pool:
1. Created SchedulingConfig struct:
- nodeSelector - Target specific nodes by labels
- affinity - Complex node/pod affinity rules
- tolerations - Schedule on tainted nodes
- topologySpreadConstraints - Distribute across failure domains
- resources - CPU/memory requests and limits
- priorityClassName - Override tenant-level priority
2. Uses #[serde(flatten)] for clean code organization:
- Groups related scheduling fields
- Maintains flat YAML structure (backward compatible)
- Follows industry pattern (MongoDB, PostgreSQL operators)
3. Enables advanced deployment patterns:
- Hardware targeting (nodeSelector for specific hardware types)
- Geographic distribution (affinity for regions/zones)
- Spot instance optimization (tolerations for spot taints)
- High availability (topology spread across zones)
- Resource differentiation (different CPU/memory per pool)
## Implementation Details
- Pool-level scheduling fields applied to StatefulSet PodSpec
- Pool-level resources applied to Container
- Pool-level priority class overrides tenant-level with fallback
- All fields optional (100% backward compatible)
- Re-exported SchedulingConfig from v1alpha1 module
## Testing
- Added 5 new tests for scheduling field propagation
- All tests passing (25/25)
- Verified node selector, tolerations, priority, resources
## Breaking Changes
None. All new fields are Option<T>, existing Tenants work unchanged.
## Verification
All changes verified against:
- RustFS source: ~/git/rustfs/rustfs/src/config/mod.rs
- RustFS constants: ~/git/rustfs/crates/config/src/constants/app.rs
- RustFS Helm chart: ~/git/rustfs/helm/rustfs/
- RustFS MNMD example: ~/git/rustfs/docs/examples/mnmd/
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Shahab96
force-pushed
the
feature/pool-scheduling-enhancements
branch
from
276811c to
736297d
Compare
Collaborator
Author
|
Latest available image used for testing |
The operator was panicking at runtime with:
"Could not automatically determine the process-level CryptoProvider from Rustls crate features"
This occurred because kube uses hyper-rustls which uses rustls, but the
rustls crypto backend (ring) wasn't being properly configured through the
dependency chain.
Solution:
- Enable rustls-tls feature on kube crate
- This ensures kube's rustls dependency gets the ring crypto provider
- The existing rustls = {version="0.23", features = ["ring"]} alone
wasn't sufficient because it doesn't affect kube's dependency tree
The rustls-tls feature on kube properly configures:
- hyper-rustls with rustls backend
- rustls with ring crypto provider
- TLS support for Kubernetes API client
This is the standard solution for kube-rs applications.
References:
- https://github.com/kube-rs/kube/blob/main/kube-client/Cargo.toml
- https://docs.rs/kube/latest/kube/#features
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds advanced multi-pool scheduling capabilities and fixes critical bugs discovered through comprehensive analysis of the RustFS source code (~/git/rustfs).
Critical Bug Fixes
Verified against RustFS source code (crates/config/src/constants/app.rs):
Fix console port: 9090 → 9001
Fix IO service port: 90 → 9000
Add required RustFS environment variables:
Standardize volume paths to RustFS convention:
Multi-Pool Scheduling Enhancements
Added comprehensive Kubernetes scheduling capabilities per pool:
Created SchedulingConfig struct:
Uses #[serde(flatten)] for clean code organization:
Enables advanced deployment patterns:
Implementation Details
Testing
Breaking Changes
None. All new fields are Option, existing Tenants work unchanged.
Verification
All changes verified against:
🤖 Generated with Claude Code