Repository navigation
Compute Test Results
Kel edited this page Sep 29, 2025
·
3 revisions
This document explains the output and interpretation of compute performance tests in the Metal Performance Tester. Compute tests measure GPU compute shader performance using parallel processing workloads.
Compute Test Configuration:
- Baseline: Low Compute Baseline
- Device: AMD Radeon Pro 5300M
- Threadgroup size: 16x16x1
- Threadgroup count: 8x8x1
- Compute workload complexity: 3/10
| Parameter | Description | Impact on Performance |
|---|---|---|
| Baseline | Reference performance data for comparison | Determines what "normal" performance looks like |
| Device | GPU hardware being tested | Different GPUs have different compute capabilities |
| Threadgroup size | Number of threads per threadgroup (x, y, z) | Affects memory access patterns and synchronization |
| Threadgroup count | Number of threadgroups to dispatch | Determines total parallel workload |
| Compute workload complexity | Algorithm complexity level (1-10) | Higher complexity = more instructions per thread |
- Size (16x16x1): 256 threads per threadgroup
- Count (8x8x1): 64 threadgroups total
- Total threads: 256 × 64 = 16,384 parallel threads
- 1-3: Simple operations (arithmetic, basic math)
- 4-6: Moderate algorithms (sorting, filtering)
- 7-10: Complex algorithms (FFT, matrix operations)
Performance Comparison:
- Current Throughput: 133.4 ops/sec
- Baseline Throughput: 130.3 ops/sec
- Throughput Change: +2.4%
- Current Throughput: Operations completed per second in current test
- Baseline Throughput: Operations per second from reference baseline
-
Throughput Change: Percentage difference between current and baseline
- Positive values: Performance improvement
- Negative values: Performance regression
- ±5%: Typically considered normal variance
- ±10%+: May indicate significant change
- Higher throughput: More work completed per unit time
- Lower throughput: Less work completed per unit time
-
Factors affecting throughput:
- Memory bandwidth utilization
- Compute unit efficiency
- Thread synchronization overhead
- Algorithm complexity
Statistical Analysis:
- Mean Difference: -0.177 ms
- Confidence Interval: [-0.641, 0.287] ms
- Statistical Significance: No
- Regression Detected: No
- Definition: Average difference in execution time between current and baseline
- Units: Milliseconds (ms)
-
Interpretation:
- Positive values: Slower execution (performance regression)
- Negative values: Faster execution (performance improvement)
- Your result: -0.177 ms (slightly faster, but not significant)
- Definition: Range where the true mean difference likely falls
-
Interpretation:
- Includes zero: No statistically significant difference
- Doesn't include zero: Statistically significant difference
- Example: [-0.641, 0.287] includes zero, so no significant change
- No: Observed difference is within normal measurement variance
- Yes: Observed difference is statistically meaningful
- No: Performance is within acceptable bounds
- Yes: Performance has significantly degraded
Quality Comparison:
- Current Quality: Fair
- Baseline Quality: Fair
Quality is based on Coefficient of Variation (CV) = Standard Deviation / Mean
| Rating | CV Range | Description | Typical Causes |
|---|---|---|---|
| Excellent | < 5% | Very consistent measurements | Ideal test conditions, stable system |
| Good | 5-10% | Good consistency | Minimal background load |
| Fair | 10-20% | Moderate variability | Normal system conditions |
| Poor | ≥ 20% | High variability | Background processes, thermal throttling |
- Higher quality: More reliable detection of compute performance changes
- Lower quality: Still functional, but may miss subtle optimizations
- Compute-specific factors: Memory bandwidth fluctuations, cache behavior
Compute Utilization Comparison:
- Compute Utilization: 38.0% (baseline: 33.0%, change: +15.0%)
- Memory Utilization: 56.7% (baseline: 56.5%, change: +0.3%)
- Total Utilization: 38.0% (baseline: 33.0%, change: +15.0%)
- Definition: Percentage of GPU compute units actively processing
- Factors: Threadgroup efficiency, instruction throughput
- Normal range: 20-80% for typical compute workloads
- Your result: 38.0% indicates moderate compute unit usage
- Definition: Percentage of memory bandwidth being used
- Factors: Memory access patterns, data transfer rates
- Normal range: 30-90% for memory-intensive workloads
- Your result: 56.7% indicates moderate memory usage
- Definition: Overall GPU utilization for compute workload
- Calculation: Weighted average based on compute vs memory workload
- Normal range: 40-90% for active compute workloads
- Small changes (±5%): Normal variance
- Medium changes (±15%): May indicate workload differences
- Large changes (±50%+): Likely configuration or system differences
NO SIGNIFICANT CHANGE: Compute performance is within normal variance.
No action required.
- Performance difference is within normal measurement variance
- No regression or improvement detected
- Continue with normal development workflow
- Compute performance has significantly degraded
- Investigate recent changes (algorithms, memory access patterns)
- Consider optimizing compute shaders or threadgroup configuration
- Compute performance has significantly improved
- Verify the improvement is intentional and stable
- Update baseline if improvement is permanent
- Optimal threadgroup size: Depends on GPU architecture
- Memory coalescing: How threads access memory together
- Synchronization overhead: Barriers and atomic operations
- Coalesced access: Efficient memory bandwidth usage
- Cache utilization: L1/L2 cache hit rates
- Memory latency: Time to fetch data from memory
- Warp/wavefront efficiency: How well threads execute together
- Instruction throughput: Instructions per clock cycle
- Resource occupancy: Utilization of compute units
- Check threadgroup size: May not match GPU architecture
- Review memory access patterns: Ensure coalesced access
- Verify workload distribution: Balance compute vs memory work
- Optimize memory access: Reduce unnecessary memory transfers
- Use local memory: Store frequently accessed data locally
- Batch operations: Reduce memory transaction overhead
- Check system load: Close background applications
- Monitor thermal conditions: Prevent throttling
- Verify consistent test conditions: Same system state
- Match GPU architecture: Use optimal threadgroup sizes
- Balance work distribution: Even workload across threadgroups
- Minimize synchronization: Reduce barriers and atomics
- Coalesce memory access: Adjacent threads access adjacent memory
- Use appropriate memory types: Local, private, global
- Minimize memory transfers: Keep data on GPU when possible
- Choose representative algorithms: Match real-world usage
- Scale appropriately: Test different workload sizes
- Profile memory usage: Monitor bandwidth utilization
- Use Metal Performance Shaders (MPS) for optimization
- Profile compute shader execution with Instruments
- Monitor memory bandwidth and cache behavior
- Optimize for GPU architecture (Apple Silicon vs discrete GPUs)
- Use vectorized operations when possible
- Minimize divergent execution paths
- Monitor GPU power consumption during compute
- Check thermal management settings
- Verify memory allocation and deallocation patterns
| Aspect | Graphics Tests | Compute Tests |
|---|---|---|
| Primary metric | FPS (frames per second) | Throughput (ops/sec) |
| Workload type | Rendering pipeline | Parallel computation |
| Utilization focus | Vertex/Fragment stages | Compute units/Memory |
| Optimization target | Frame time consistency | Instruction throughput |
| Memory patterns | Texture/vertex data | Structured data arrays |
- Set performance thresholds: Typically ±10% for compute workloads
- Monitor trends: Track performance over time
- Alert on regressions: Integrate with build systems
- Update baselines: When performance improvements are intentional
- Version control: Track baseline changes with code changes
- Environment consistency: Ensure test environments match production