Skip to content

Testing: Add performance and load testing suite #62

@sfloess

Description

@sfloess

Overview

Add performance benchmarks and load testing to ensure JNexus performs well under stress and to catch performance regressions.

Current State

  • No performance testing
  • No load testing
  • No benchmarks for regression detection
  • Performance characteristics unknown

Testing Score Impact

Current: Testing A- (90/100)
With this: Testing A+ (98/100)

Proposed Tests

1. Benchmark Suite (JMH)

Add Java Microbenchmark Harness for critical paths:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.37</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.37</version>
    <scope>test</scope>
</dependency>

Benchmarks to add:

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
public class NexusClientBenchmark {
    
    @Benchmark
    public void listComponents_cached() {
        // Measure cache hit performance
    }
    
    @Benchmark
    public void listComponents_uncached() {
        // Measure cache miss + HTTP roundtrip
    }
    
    @Benchmark
    public void parseJsonResponse_1000components() {
        // Measure JSON parsing overhead
    }
    
    @Benchmark
    public void regexValidation_complexPattern() {
        // Measure ReDoS prevention overhead
    }
}

2. Load Testing

Test with realistic workloads:

Scenario 1: Large Repository

  • 10,000 components
  • Pagination (10 pages @ 1000 items each)
  • Verify memory usage stays <500MB

Scenario 2: Concurrent Operations

  • 5 simultaneous list operations
  • Verify thread safety
  • No deadlocks or race conditions

Scenario 3: Bulk Delete

  • Delete 1,000 components
  • Verify progress callback performance
  • Measure time per component

3. Stress Testing

Cache Stress:

@Test
void testCacheUnder10000Repositories() {
    // Fill cache with 10K repos
    // Verify memory footprint
    // Verify LRU eviction works
}

Connection Stress:

@Test
void testRetryLogicUnderServerStress() {
    // Simulate flaky server (50% failure rate)
    // Verify retry logic holds up
    // Measure total time with backoff
}

4. Memory Profiling

Add heap dump analysis on large operations:

java -Xmx512m -XX:+HeapDumpOnOutOfMemoryError \
  -jar jnexus.jar list large-repo --show-metadata

5. Performance Assertions

Set performance SLAs in tests:

@Test
@Timeout(value = 5, unit = TimeUnit.SECONDS)
void listComponents_shouldCompleteIn5Seconds() {
    // Fail if >5s for 1000 components
}

Acceptance Criteria

Performance Targets:

  • Cache hit: <10ms
  • Cache miss + HTTP: <500ms (p95)
  • JSON parsing 1000 items: <100ms
  • Regex validation: <5ms
  • Memory: <200MB for 10K components

Load Targets:

  • Handle 10K components without OOM
  • 5 concurrent operations without deadlock
  • Delete 1000 items in <5 minutes

CI Integration

Add to GitHub Actions (optional):

- name: Run Benchmarks
  run: mvn test -Pbenchmark
  
- name: Upload Benchmark Results
  uses: benchmark-action/github-action-benchmark@v1
  with:
    tool: 'jmh'
    output-file-path: target/jmh-result.json

Benefits

  • Catch performance regressions in CI
  • Establish baseline metrics
  • Validate scalability claims
  • Prove efficiency (cache hit rate, etc.)

Priority

Medium - Important for production confidence

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions