QWK.NET ‐ QWK Packet Reference

QWK Packet Reference

QWK Message Header Specification

Complete 128-Byte Header Layout

Offset  Length  Field               Format          Validation
------  ------  ------------------  --------------  ---------------------------
0       1       Status byte         ASCII char      0x20-0x7E (printable)
1       7       Message number      ASCII spaces    Optional: numeric
8       8       Date                MM-DD-YY        Hyphens at [2] and [5]
16      5       Time                HH:MM           Colon at [2]
21      25      To (recipient)      ASCII/CP437     Space-padded
46      25      From (sender)       ASCII/CP437     Space-padded
71      25      Subject             ASCII/CP437     Space-padded
96      12      Password            ASCII spaces    Rarely used
108     8       Reference number    ASCII spaces    Numeric (reply-to)
116     6       Block count         ASCII spaces    Right-aligned numeric
122     1       Alive flag          Binary          0xE1 or 0xE2
123     2       Conference number   Little-endian   Unsigned short
125     2       Logical record      Little-endian   Unsigned short
127     1       Network tag         Binary          0x20, 0x2A, or 0xFF

Critical Validation Fields

The following fields are highly reliable for distinguishing headers from body blocks:

1. Date Field (bytes 8-15)

Standard Format: MM-DD-YY (8 characters, per QWK specification)

Library-Supported Variants: The library accepts multiple real-world formats:

• MM-DD-YY (hyphen, 2-digit year) - QWK spec standard
• MM/DD/YY (slash, 2-digit year) - Common BBS variant
• MM-DD-YYYY (hyphen, 4-digit year) - Extended format
• MM/DD/YYYY (slash, 4-digit year) - Extended variant (e.g., mvt2.qwk)

Validation for Header Detection:

• Byte 10 (position 2): Must be '-' (0x2D) OR '/' (0x2F)
• Byte 13 (position 5): Must be '-' (0x2D) OR '/' (0x2F)
• Delimiters must be consistent (both hyphens OR both slashes)

Why reliable for distinguishing headers from body blocks:

• Body text rarely has two delimiters in exactly these positions
• Even if body contains dates, unlikely to be at offset 8-15
• Combined with other checks, creates strong validation
• Accepting both '-' and '/' increases real-world compatibility

Examples:

Valid:   "06-15-93"  → bytes[10]='-', bytes[13]='-' ✓ (spec standard)
Valid:   "12/31/99"  → bytes[10]='/', bytes[13]='/' ✓ (slash variant)
Valid:   "08/01/1994" → bytes[10]='/', bytes[13]='/' ✓ (4-digit year)
Invalid: "June 15 "  → bytes[10]='e', bytes[13]='1' ✗
Invalid: "ππππππππ"  → bytes[10]='π', bytes[13]='π' ✗
Invalid: "06.15.93"  → bytes[10]='.', bytes[13]='.' ✗ (dots not supported)

Important Note: The header validation is for distinguishing headers from body blocks, not for full date parsing. The actual date parsing (in QwkMessageHeader.TryGetDateTime()) handles these variants properly with Y2K heuristics (00-49→2000s, 50-99→1900s) and 4-digit year validation (1980-2099).

2. Time Field (bytes 16-20)

Format: HH:MM (5 characters)

Position:  0  1  2  3  4
Value:     H  H  :  M  M
Byte:      16 17 18 19 20

Validation:

• Byte 18 (position 2): Must be ':' (0x3A)

Why reliable:

• Colon at exact position 18 is uncommon in body text
• Even if body contains times, unlikely at offset 16-20
• Very strong signal when combined with date hyphens

Examples:

Valid:   "22:25"  → bytes[18]=':' ✓
Valid:   "09:00"  → bytes[18]=':' ✓
Invalid: "night"  → bytes[18]='g' ✗
Invalid: "ππππ"   → bytes[18]='π' ✗

3. Status Byte (byte 0)

Format: Single ASCII character

Valid values:

' ' (0x20) = Public message, unread
'-' (0x2D) = Public message, read
'*' (0x2A) = Private message, unread
'+' (0x2B) = Private message, read
'~' (0x7E) = Comment to sysop, unread
'`' (0x60) = Comment to sysop, read
'%' (0x25) = Password protected (sender), unread
'^' (0x5E) = Password protected (sender), read
'!' (0x21) = Password protected (group), unread
'#' (0x23) = Password protected (group), read
'$' (0x24) = Password protected (group to all)

Validation:

• Must be in range 0x20-0x7E (printable ASCII)
• Body blocks often start with letters or 0xE3 line terminators

Why reliable:

• Body text starting with control characters (< 0x20) is common
• Body text starting with high ASCII (> 0x7E) is common in BBS messages
• Headers always have printable status bytes

Examples:

Valid:   0x2A ('*')  ✓ Private message
Valid:   0x20 (' ')  ✓ Public message
Invalid: 0x1B        ✗ ESC character (ANSI codes in body)
Invalid: 0xE3        ✗ Line terminator (body text)
Invalid: 0x00        ✗ Null (padding in body)

4. Alive Flag (byte 122)

Format: Single byte

Valid values:

• 0xE1 = Message is alive (active)
• 0xE2 = Message is killed (deleted)

Validation:

• Must be exactly 0xE1 or 0xE2
• No other values are valid per QWK specification

Why reliable:

• Body text at byte 122 is essentially random
• Probability of exactly 0xE1 or 0xE2 is ~0.8% (2/256)
• Very strong discriminator

Examples:

Valid:   0xE1  ✓ Active message
Valid:   0xE2  ✓ Deleted message
Invalid: 0x20  ✗ Space (common in body padding)
Invalid: 0x61  ✗ 'a' (common in body text)
Invalid: 0xE3  ✗ Line terminator (very common in bodies)

---

Validation Implementation

Recommended Approach

Use all four checks in combination for maximum reliability:

private static bool IsPlausibleMessageHeader(ReadOnlySpan<byte> headerBytes)
{
  // Check 1: Correct size
  if (headerBytes.Length != 128)
    return false;

  // Check 2: Status byte in printable ASCII range
  byte statusByte = headerBytes[0];
  if (statusByte < 0x20 || statusByte > 0x7E)
    return false;

  // Check 3: Date field has delimiters (hyphen OR slash) at positions 2 and 5
  // Date is at bytes 8-15, so delimiters are at absolute positions 10 and 13
  // Accept both '-' (0x2D) and '/' (0x2F) as valid delimiters
  byte delimiter1 = headerBytes[10];
  byte delimiter2 = headerBytes[13];
  
  // Must be hyphens OR slashes (but consistent)
  bool hasHyphens = (delimiter1 == (byte)'-' && delimiter2 == (byte)'-');
  bool hasSlashes = (delimiter1 == (byte)'/' && delimiter2 == (byte)'/');
  
  if (!hasHyphens && !hasSlashes)
    return false;

  // Check 4: Time field has colon at position 2
  // Time is at bytes 16-20, so colon is at absolute position 18
  if (headerBytes[18] != (byte)':')
    return false;

  // Check 5: Alive flag is valid
  byte aliveFlag = headerBytes[122];
  if (aliveFlag != 0xE1 && aliveFlag != 0xE2)
    return false;

  // All checks passed
  return true;
}

Statistical Reliability

Probability of false positive (body block passing validation):

P(status OK)     = 95/256   ≈ 37%   (95 printable ASCII chars)
P(date delim 1)  = 2/256    ≈ 0.8%  (hyphen OR slash)
P(date delim 2)  = 2/256    ≈ 0.8%  (hyphen OR slash)
P(time colon)    = 1/256    ≈ 0.4%
P(alive flag)    = 2/256    ≈ 0.8%

P(all pass) = 0.37 × 0.008 × 0.008 × 0.004 × 0.008
            ≈ 0.000000075
            ≈ 1 in 13 million

Conclusion: False positive rate is negligible. A body block is extremely unlikely to pass all five checks, even when accepting both date delimiter types.

---

Edge Cases and Special Situations

1. Date Format Variations

The library supports multiple real-world date format variants:

Supported formats:

MM-DD-YY     ✓ Hyphens, 2-digit year (QWK spec standard)
MM/DD/YY     ✓ Slashes, 2-digit year (common BBS variant)
MM-DD-YYYY   ✓ Hyphens, 4-digit year (extended format)
MM/DD/YYYY   ✓ Slashes, 4-digit year (e.g., mvt2.qwk historical packet)

Validation accepts:

• Both hyphen '-' (0x2D) and slash '/' (0x2F) delimiters
• Delimiters must be consistent (both positions must match)
• 4-digit years validated in range 1980-2099
• 2-digit years use Y2K heuristic: 00-49→2000s, 50-99→1900s

Historical variants NOT supported (will be skipped):

06.15.93   ✗ Dots instead of hyphens/slashes (INVALID)
15-06-93   ✗ DD-MM-YY European format (may accidentally pass validation)
Jun-15-93  ✗ Month name abbreviation (INVALID)

Recommendation:

• Accept hyphens OR slashes at validation positions 10 and 13
• Delimiters must be consistent (both same type)
• Full date parsing handles Y2K heuristics and 4-digit year validation
• In lenient mode, log when skipping non-standard formats

2. Body Blocks That Look Like Headers

Scenario: Body text coincidentally has valid date/time patterns

Example:

"Meeting scheduled for 06-15-93 at 10:30 to discuss..."

Why this still fails validation:

• Status byte would need to be printable (might pass)
• Date hyphens would need to be at exact byte positions 10 and 13 (unlikely)
• Time colon would need to be at exact byte position 18 (unlikely)
• Alive flag at byte 122 would need to be 0xE1 or 0xE2 (very unlikely)

Probability: ~1 in 53 million (see statistical analysis above)

3. Corrupted Headers

Scenario: Actual message header has corrupted date/time field

Example:

Offset 0:  0x2A  (valid status)
Offset 10: 0x00  (corrupted, should be '-')
Offset 13: 0x2D  (valid hyphen)
Offset 18: 0x3A  (valid colon)
Offset 122: 0xE1 (valid alive flag)

Result: Fails validation (missing first hyphen), message skipped

Handling:

• Log warning: "Skipping potentially corrupted header at offset X"
• Continue to next block
• This is correct behaviour for preservation-grade software
• Better to skip one corrupted message than misparse entire packet

4. Very Short Messages (Block Count = 1)

Scenario: Message header with no body blocks

Block count = 1  (header only, no body)

Parsing flow:

1. Read 128 bytes → validate as header ✓
2. Parse header: BlockCount = 1
3. Calculate body blocks: 1 - 1 = 0
4. Read 0 body blocks
5. Create message with empty body
6. Next iteration reads next block → validate as header

Result: Works correctly (no issue)

5. Huge Block Counts (Potential DoS)

Scenario: Malicious or corrupted header claims 10,000 blocks

Block count = 10000  (1.25 MB for one message)

Handling:

• Current code will attempt to read 10,000 blocks
• If blocks don't exist (EOF), logs warning and stops
• No infinite loop risk (stream exhaustion stops parsing)

Future enhancement (optional):

const int MaxReasonableBlockCount = 1000; // ~128 KB

if (header.BlockCount > MaxReasonableBlockCount)
{
  context.AddWarning(
    $"Message {messageNumber} claims {header.BlockCount} blocks " +
    $"(>{MaxReasonableBlockCount * 128} bytes). This may indicate corruption.");
  
  // In strict mode, could throw
  // In lenient mode, could cap the read or skip the message
}

---

Testing Data Patterns

Valid Header Examples

Example 1: Typical personal message

Offset 0:   0x2A  ('*' = private, unread)
Offset 1-7: "     1 "  (message number 1)
Offset 8-15: "06-15-93"  (June 15, 1993)
Offset 16-20: "22:25"  (10:25 PM)
Offset 21-45: "JOHN DOE" + spaces
Offset 46-70: "JANE SMITH" + spaces
Offset 71-95: "Test Message" + spaces
Offset 116-121: "     2"  (2 blocks total)
Offset 122: 0xE1  (alive)
Offset 123-124: 0x0000  (conference 0, little-endian)

Validation: ✓ All checks pass

Example 2: Public read message

Offset 0:   0x2D  ('-' = public, read)
Offset 8-15: "12-31-99"
Offset 16-20: "23:59"
Offset 122: 0xE1

Validation: ✓ All checks pass

Invalid Body Block Examples

Example 1: Body text with 0xE3 terminators

Offset 0:   0xE3  (line terminator, not printable ASCII)
Offset 10:  0x65  ('e' in "message")
Offset 13:  0x20  (space)
Offset 18:  0x74  ('t' in "text")
Offset 122: 0x20  (space padding)

Validation: ✗ Fails status byte check (0xE3 < 0x20)

Example 2: Body text with box-drawing characters

Offset 0:   0xB3  (│ box character, > 0x7E)
Offset 10:  0xC4  (─ box character)
Offset 13:  0xBF  (┐ box character)
Offset 18:  0xB3  (│ box character)
Offset 122: 0xC4  (─ box character)

Validation: ✗ Fails status byte check (0xB3 > 0x7E)

Example 3: English text body

Offset 0:   0x54  ('T' in "This is...")
Offset 8-15: "his is a"  (no hyphens at 10, 13)
Offset 16-20: " mess"  (no colon at 18)
Offset 122: 0x67  ('g' in "something")

Validation: ✗ Fails date hyphen check AND time colon check

---

Performance Considerations

Validation Cost

Each validation consists of:

• 1× length check (branch)
• 4× byte comparisons (cheap)
• 1× byte range check (two comparisons)

Total: ~7 comparisons per block

Cost: Negligible compared to:

• Stream I/O (reading 128 bytes)
• Memory allocation
• CP437 decoding
• String construction

Recommendation: Always validate, performance impact is unmeasurable.

Memory Efficiency

Use ReadOnlySpan<byte> for zero-allocation validation:

// Good: No allocation
byte[] headerBytes = new byte[128];
stream.Read(headerBytes, 0, 128);
bool isValid = IsPlausibleMessageHeader(headerBytes);  // ReadOnlySpan<byte> implicit cast

// Also good: Explicit span
ReadOnlySpan<byte> span = headerBytes.AsSpan();
bool isValid = IsPlausibleMessageHeader(span);

---

Summary

Critical validation fields:

1. Status byte: 0x20-0x7E
2. Date hyphens: bytes[10]='-', bytes[13]='-'
3. Time colon: bytes[18]=':'
4. Alive flag: 0xE1 or 0xE2

Implementation:

• Simple, fast byte comparisons
• No allocations
• ~1 in 53 million false positive rate
• Handles malformed packets gracefully

Testing:

• Test with real-world malformed packets
• Test with synthetic corrupted data
• Verify round-trip preservation