Skip to content

QWK.NET ‐ Benchmarking

Jump to bottom
Agent 57951 edited this page Jan 19, 2026 · 1 revision

QWK.NET Benchmarking - Memory Profiling Guide

Overview

The QwkNet.Benchmarking tool includes comprehensive memory profiling capabilities to track allocation patterns and verify efficient memory usage across different packet sizes.

Memory Profiling Features

1. Thread-Local Allocation Tracking

The benchmark tool uses GC.GetAllocatedBytesForCurrentThread() to measure precise memory allocations for each parsing phase:

  • Archive Reader: Memory overhead from opening and managing ZIP archives
  • CONTROL.DAT: Allocations during BBS metadata parsing
  • Message Collection: Primary memory consumer, scales with message count
  • Optional Files: Allocations from enumerating additional packet files

2. Peak Memory Monitoring

During message parsing, the tool samples peak memory usage every 10 messages to capture the high-water mark without excessive profiling overhead.

3. Component Breakdown

Memory allocations are categorised by component with percentage distribution, allowing developers to identify which parts of the library consume the most memory.

4. Per-Message Metrics

The tool calculates average memory overhead per message, enabling validation of linear scaling behaviour.

Expected Memory Usage Patterns

Memory usage will vary based on actual packet contents. The benchmark tool measures:

Example Small Packet (50-100 messages)

When you run the benchmark on a small packet, you'll see output like:

Memory Usage:
┌──────────────────────┬─────────────┐
│ Metric               │ Value       │
├──────────────────────┼─────────────┤
│ Heap Allocated       │     X.XX MB │  ← Actual heap delta
│ Peak Memory          │     X.XX MB │  ← Maximum memory during parsing
│ Thread Allocated     │     X.XX MB │  ← Total thread-local allocations
│ Per Message          │    XX.XX KB │  ← Message Collection / Message Count
└──────────────────────┴─────────────┘

Memory Allocation by Component:
┌──────────────────────┬─────────────┬──────────┐
│ Component            │ Allocated   │ Percent  │
├──────────────────────┼─────────────┼──────────┤
│ Archive Reader       │     X.XX MB │     X.X% │  ← Should be small, fixed
│ CONTROL.DAT          │     X.XX MB │     X.X% │  ← Should be small, fixed
│ Message Collection   │     X.XX MB │    XX.X% │  ← Should dominate (70-95%)
│ Optional Files       │     X.XX MB │     X.X% │  ← Should be small, fixed
└──────────────────────┴─────────────┴──────────┘

Key Verification Points:

  • Message Collection should be 70-95% of total allocations
  • Archive Reader, CONTROL.DAT, and Optional Files should each be <15% individually
  • Per Message overhead should remain consistent across different packet sizes

Example Large Packet (500-1000 messages)

For larger packets, the component percentages should shift:

Memory Allocation by Component:
┌──────────────────────┬─────────────┬──────────┐
│ Component            │ Allocated   │ Percent  │
├──────────────────────┼─────────────┼──────────┤
│ Archive Reader       │     X.XX MB │     X.X% │  ← Smaller % (fixed cost)
│ CONTROL.DAT          │     X.XX MB │     X.X% │  ← Smaller % (fixed cost)
│ Message Collection   │    XX.XX MB │    XX.X% │  ← Larger % (scales with N)
│ Optional Files       │     X.XX MB │     X.X% │  ← Smaller % (fixed cost)
└──────────────────────┴─────────────┴──────────┘

Expected Trend:

  • Fixed components (Archive, CONTROL.DAT, Optional) have smaller percentages in large packets
  • Message Collection has larger percentage in large packets (approaching 95-98%)
  • Absolute values of fixed components should remain similar to small packets

Linear Scaling Verification

The benchmark tool enables you to verify linear scaling by testing packets of different sizes.

Measurement Methodology

Step 1: Test Multiple Packet Sizes

./QwkNet.Benchmarking benchmark packet_50msg.qwk --iterations=5
./QwkNet.Benchmarking benchmark packet_100msg.qwk --iterations=5
./QwkNet.Benchmarking benchmark packet_500msg.qwk --iterations=5
./QwkNet.Benchmarking benchmark packet_1000msg.qwk --iterations=5

Step 2: Record Fixed Components

For each packet size, note the allocations for:

  • Archive Reader (MB)
  • CONTROL.DAT (MB)
  • Optional Files (MB)

These should remain similar across all packet sizes (within ±20%).

Step 3: Record Per-Message Overhead

Note the "Per Message" value in KB for each packet size.

This should remain consistent across all packet sizes (within ±20%).

Step 4: Verify Linear Formula

Expected Total = Fixed Components + (Per-Message × Message Count)

Plot your measurements:

Packet Size → Total Thread Allocated
50 messages → X.XX MB
100 messages → Y.YY MB
500 messages → Z.ZZ MB
1000 messages → A.AA MB

If the points form a straight line, scaling is linear.

Example Verification

Fixed Components (should be constant):

50-message packet:   Archive=0.45MB, CONTROL.DAT=0.12MB, Optional=0.15MB → Total Fixed=0.72MB
100-message packet:  Archive=0.46MB, CONTROL.DAT=0.12MB, Optional=0.15MB → Total Fixed=0.73MB
500-message packet:  Archive=0.48MB, CONTROL.DAT=0.13MB, Optional=0.16MB → Total Fixed=0.77MB
1000-message packet: Archive=0.49MB, CONTROL.DAT=0.13MB, Optional=0.17MB → Total Fixed=0.79MB

✅ Fixed components vary by <10% - this is good.

Per-Message Overhead (should be constant):

50-message packet:   50.2 KB/message
100-message packet:  51.1 KB/message
500-message packet:  52.0 KB/message
1000-message packet: 51.8 KB/message

✅ Per-message varies by <4% - this confirms linear scaling.

Total Memory:

50 messages:   0.72 MB + (50 × 0.050 MB) = 3.22 MB (predicted) vs 3.25 MB (actual)
100 messages:  0.73 MB + (100 × 0.051 MB) = 5.83 MB (predicted) vs 5.87 MB (actual)
500 messages:  0.77 MB + (500 × 0.052 MB) = 26.77 MB (predicted) vs 26.85 MB (actual)
1000 messages: 0.79 MB + (1000 × 0.052 MB) = 52.79 MB (predicted) vs 52.94 MB (actual)

✅ Predicted values match actual within 1-2% - linear scaling confirmed.

Usage Examples

Basic Memory Profiling

# Profile a typical QWK packet
./QwkNet.Benchmarking benchmark packet.qwk

# Profile with multiple iterations for statistical accuracy
./QwkNet.Benchmarking benchmark packet.qwk --iterations=10

# Profile in strict validation mode
./QwkNet.Benchmarking benchmark packet.qwk --mode=Strict

Comparing Memory Usage Across Packet Sizes

# Small packet (50 messages)
./QwkNet.Benchmarking benchmark small.qwk --iterations=5

# Medium packet (200 messages)
./QwkNet.Benchmarking benchmark medium.qwk --iterations=5

# Large packet (1000 messages)
./QwkNet.Benchmarking benchmark large.qwk --iterations=5

The per-message memory overhead should remain consistent across all packet sizes, confirming linear scaling.

Interpreting Results

Healthy Memory Patterns

✓ Archive Reader: 5-10% of total allocations (fixed overhead) ✓ CONTROL.DAT: <5% of total allocations (small fixed overhead) ✓ Message Collection: 80-95% of total allocations (primary consumer) ✓ Optional Files: <5% of total allocations (minimal overhead) ✓ Per-Message Overhead: 40-60 KB per message (consistent across packet sizes)

Warning Signs

⚠ Archive Reader >20%: Excessive archive handling overhead ⚠ Message Collection <70%: Unexpected allocations in fixed components ⚠ Per-Message Overhead >100 KB: Inefficient message parsing ⚠ Per-Message Overhead varies significantly: Non-linear memory growth

Technical Implementation Details

Baseline Memory Measurement

// Force full GC to establish accurate baseline
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
long memoryBefore = GC.GetTotalMemory(false);
long threadAllocatedBefore = GC.GetAllocatedBytesForCurrentThread();

Component-Level Tracking

// Track allocations for each parsing phase
long archiveAllocBefore = GC.GetAllocatedBytesForCurrentThread();
QwkPacket packet = QwkPacket.Open(packetPath, mode);
long archiveAllocAfter = GC.GetAllocatedBytesForCurrentThread();
result.ArchiveAllocatedBytes = archiveAllocAfter - archiveAllocBefore;

Peak Memory Sampling

// Sample peak memory every 10 messages to reduce profiling overhead
if (messageCount % 10 == 0)
{
  long currentMemory = GC.GetTotalMemory(false);
  if (currentMemory > peakMemoryDuringParsing)
  {
    peakMemoryDuringParsing = currentMemory;
  }
}

Memory Efficiency Best Practices

For Library Developers

  1. Minimize Fixed Overhead: Keep archive reader and metadata parsing allocations minimal
  2. Lazy Evaluation: Parse messages on-demand rather than eagerly loading all data
  3. Reuse Buffers: Consider pooling byte buffers for message body parsing
  4. Avoid String Duplication: Store raw bytes where appropriate, decode on-demand

For Library Users

  1. Process Incrementally: Enumerate messages one at a time rather than loading all into memory
  2. Dispose Promptly: Use using statements to ensure timely resource cleanup
  3. Consider Streaming: For very large packets, process messages in batches
  4. Monitor Memory: Use the benchmark tool to validate memory usage for your workloads

Performance Validation Methodology

Use the benchmark tool to validate that your specific packets meet reasonable performance targets:

Recommended Testing Approach

1. Establish Your Baseline

# Test with your typical packet sizes
./QwkNet.Benchmarking benchmark typical_packet.qwk --iterations=10

Record the results:

  • Total parse time (ms)
  • Thread allocated memory (MB)
  • Per-message overhead (KB)

2. Test Larger Packets

# Test with larger packets to verify scaling
./QwkNet.Benchmarking benchmark large_packet.qwk --iterations=10

Verify:

  • Per-message time remains similar
  • Per-message memory remains similar
  • Total time grows linearly with message count

3. Compare Across Validation Modes

./QwkNet.Benchmarking benchmark packet.qwk --mode=Strict --iterations=5
./QwkNet.Benchmarking benchmark packet.qwk --mode=Lenient --iterations=5
./QwkNet.Benchmarking benchmark packet.qwk --mode=Salvage --iterations=5

Note any significant differences in memory usage between modes.

Performance Expectations

The library targets sub-100ms parse times for typical packets (<100 messages).

What to Look For:

  • Parse time < 100ms for packets with <100 messages
  • Per-message time consistent across packet sizes (within ±20%)
  • Per-message memory consistent across packet sizes (within ±20%)
  • Message Collection dominates memory allocation (70-95%)
  • No excessive allocations in fixed components

Warning Signs:

  • Parse time > 100ms for small packets (<100 messages)
  • Per-message time increases significantly with packet size
  • Per-message memory varies by >30% across packet sizes
  • Archive Reader or CONTROL.DAT uses >20% of total memory
  • Peak memory exceeds thread allocated by >2x

Troubleshooting

High Memory Usage

If memory usage exceeds expectations:

  1. Check Packet Size: Verify message count matches expectations
  2. Examine Message Bodies: Large message bodies significantly impact memory
  3. Review Validation Mode: Strict mode may allocate more for error tracking
  4. Profile Components: Use component breakdown to identify the source

Non-Linear Scaling

If per-message memory varies significantly:

  1. Message Size Distribution: Check if messages have highly variable body sizes
  2. Kludge Count: Messages with many QWKE kludges consume more memory
  3. Encoding Issues: CP437 decoding overhead may vary with content
  4. Collection Overhead: Verify List<T> resize behaviour isn't causing issues

Clone this wiki locally