Repository navigation
Math Boundary Gap and Entropy
Quick-Scan Infobox
Aspect Details Target User Educators addressing model ambiguity, developers adjusting diagnostics—statistics and machine learning basics assumed. Core Goal Explain how narrow utility differences and elevated softmax entropy detect decision edges, serving as internal uncertainty measures. Time to First Insight 3 minutes: Process a close-decision prompt, search the trace for boundary indicators. Key Analogy A narrow vote margin: Minimal separation between options or spread-out preferences calls for further review rather than quick acceptance. Compitum Integration Detection in boundary module; appears in certificates to support traceable adjustments. Known Limitation Thresholds depend on candidate count; low counts may underflag—normalize by log k for consistency.
This process starts with candidate utilities and ends with a flag if conditions indicate ambiguity.
graph TD
A[Utility Vector U] --> B[Sort Descending]
B --> C[Gap g = U1 - U2]
B --> D["Softmax p = exp(tau U) / sum exp(tau U)"]
D --> E["Entropy H = - sum p_i log p_i"]
C --> F{g < delta_g?}
E --> G{H > delta_h OR unc_winner > 0.1?}
F --> H[Boundary Evaluation]
G --> H
H -->|Both True| I[Flag: Boundary Detected]
H -->|Otherwise| J[Clear Decision]
I --> K[Initiate Exploration: Sample or Reroute]
J --> L[Select Top Option]
style I fill:#fff3e0,stroke:#ef6c00
style J fill:#e8f5e9,stroke:#1b5e20
style K fill:#e3f2fd,stroke:#01579b
style A fill:#fce4ec,stroke:#c2185b
Caption: Steps from input to flag or selection. Alt text: Graph with 14 nodes: utilities to sorting, gap/entropy computation, checks, and outcomes, using amber for flag, green for clear, blue for exploration, pink for input.
A flag prompts caution: Small gaps risk errors; high entropy shows dispersed options. Review these to refine thresholds.
Visuals clarify regions: Clear areas have separation; boundaries show clustering.
graph LR
subgraph Clear Decision
A1[U₁ = 0.95] -.->|Gap 0.35| A2[U₂ = 0.60]
A3[Entropy H ≈ 0.4]
end
subgraph Boundary Area
B1[U₁ = 0.80] -.->|Gap 0.05| B2[U₂ = 0.75]
B3[Entropy H ≈ 1.2]
B4[Review Prompted]
end
subgraph Uniform Spread
C1[U₁ = 0.70]
C2[U₂ = 0.69]
C3[U₃ = 0.68]
C4[Entropy H ≈ log k]
C5[Full Review Needed]
end
style Clear fill:#e8f5e9,stroke:#1b5e20
style Boundary fill:#fff3e0,stroke:#ef6c00
style Uniform fill:#fce4ec,stroke:#c2185b
Caption: Three regions with example utilities and entropy. Alt text: Graph with three subgraphs: clear (2 edges, low H), boundary (2 edges, medium H), uniform (3 nodes, high H), using green, amber, pink fills.
In clear spaces, proceed; boundaries suggest checks. Limitation: Visuals simplify; real spaces curve with metrics.
Boundaries occur where utilities align closely, indicating selection risk: Narrow gaps between leaders suggest sensitivity to noise, while softmax entropy measures probability spread—both estimate uncertainty without external labels.
flowchart TB
subgraph Margin Roots
A1["Margin gamma = 2 / ||w||"]
A2[Maximize Separation]
A3[Generalization Aid]
end
subgraph Entropy Base
B1["H = - sum p_i log p_i"]
B2[Expected Information]
B3[Uncertainty Measure]
end
subgraph Compitum Use
C1["Gap g = U1 - U2"]
C2["Softmax p = exp(tau U) / Z"]
C3[Joint Check]
C4["Flag: g < delta_g AND H > delta_h"]
end
A3 -.-> C1
B3 -.-> C2
C1 --> C3
C2 --> C3
C3 --> C4
style C4 fill:#fff3e0,stroke:#ef6c00
Caption: Links from foundations to implementation. Alt text: Flowchart with three subgraphs (margin, entropy, Compitum), 12 nodes, arrows connecting concepts, amber for flag.
Definition:
- Utilities: U = [U₁ ≥ U₂ ≥ ... ≥ Uₖ]
- Gap: g = U₁ - U₂
- Probabilities: pᵢ = exp(τ Uᵢ) / Σ exp(τ Uⱼ) (τ=1 standard)
- Entropy: H(p) = -Σ pᵢ log pᵢ
- Flag: g < δ_g (0.05) AND (H > δ_h (1.0) OR unc_winner > 0.1)
Historical Context:
- Cortes and Vapnik (1995): Support-vector networks emphasize margins for robustness.
- Shannon (1948): Entropy as average surprise in sources.
Key Results:
- Boundaries align with constraint boundaries {x | A x = b}.
- High H links to shadow price variation >0.1.
- Predictor uncertainty boosts flag accuracy. Limitation: Assumes sorted utilities; unsorted inputs need preprocessing.
Code Structure:
sequenceDiagram
participant Router
participant Selector
participant Boundary
participant Predictor
participant Explorer
Router->>Selector: Utilities U
Selector->>Boundary: Evaluate boundary(U, unc)
Boundary->>Boundary: Gap and entropy
alt Gap < δ_g AND H > δ_h
Boundary->>Predictor: Winner uncertainty
Predictor-->>Boundary: unc_winner
alt unc_winner > 0.1
Boundary-->>Selector: Flag True
Selector->>Explorer: Exploration
Explorer-->>Router: Sample or reroute
else
Boundary-->>Selector: Flag False
end
else
Boundary-->>Selector: Flag False
end
Selector-->>Router: Selection and details
Caption: Sequence for boundary check. Alt text: Sequence diagram with five participants, 12 messages, branching on conditions, green for clear, amber for flag.
Python Implementation:
Open on GitHub | Open in nbviewer | Launch in Binder
from compitum.boundary import BoundaryAnalyzer
ba = BoundaryAnalyzer() # gap_threshold=0.05, entropy_threshold=0.65, sigma_threshold=0.12
# utilities and per-model uncertainty (sigma) are keyed by model name
U_tie = {"m1": 0.80, "m2": 0.75, "m3": 0.60}
sig_tie = {"m1": 0.15}
print("Tie:", ba.analyze(U_tie, sig_tie))
U_clear = {"m1": 0.95, "m2": 0.60, "m3": 0.50}
sig_clear = {"m1": 0.02}
print("Clear:", ba.analyze(U_clear, sig_clear))
U_gap_low_sigma = {"m1": 0.85, "m2": 0.80, "m3": 0.65}
sig_low = {"m1": 0.05}
print("Gap+entropy present, sigma too low:", ba.analyze(U_gap_low_sigma, sig_low))Note: Outputs vary with inputs; flag true on tie due to gap=0.05 and entropy>1.0 threshold.
Limitation: Computation assumes numpy/scipy; high k increases time—cap at 10 for efficiency.
stateDiagram-v2
[*] --> Count_Votes
Count_Votes --> Close_Margin: Gap < 2%
Count_Votes --> Wide_Margin: Gap > 10%
Close_Margin --> Review: Elevated Uncertainty
Close_Margin --> Audit: Divided Support
Wide_Margin --> Confirm: Low Uncertainty
Review --> Extended_Check
Audit --> Extended_Check
Extended_Check --> Boundary_Indicator
Confirm --> Final_Result
note right of Close_Margin
Narrow difference
Spread preferences
= EDGE CASE
end note
note right of Wide_Margin
Substantial lead
Concentrated choice
= DECISIVE
end note
Caption: State transitions for vote-like decisions. Alt text: State diagram with initial count branching to close/wide, then to checks or confirm, green for decisive, amber for edge.
Classroom Applications:
- Discussion Routing: Assign high-entropy prompts to group audits.
- Learning Opportunities: Treat flags as prompts for deeper inquiry.
- Group Dynamics: Entropy as discussion spread—peaked means agreement, uniform means diverse input.
Montessori Parallel: Like knobbed cylinders, a flag reveals mismatch (close fit but uncertain grasp)—child adjusts grip, system reroutes, both gaining precision through trial.
Common Patterns:
| Case | Gap | Entropy | Flag | Review Focus |
|---|---|---|---|---|
| Decisive | 0.35 | 0.6 | No | Alignment strengths |
| Narrow | 0.03 | 1.4 | Yes | Option exploration |
| Dispersed | 0.15 | 1.8 | Yes | Constraint tightening |
| Narrow but Stable | 0.04 | 0.7, unc=0.02 | No | Confidence confirmation |
Limitation: Patterns assume τ=1; vary for task sensitivity.
| Metric | Formula | Threshold | Meaning | Certificate Key |
|---|---|---|---|---|
| Gap | g = U₁ - U₂ | <0.05 | Close leaders | boundary.gap |
| Entropy | H = -Σ pᵢ log pᵢ | >1.0 (k=5) | Spread probabilities | boundary.entropy |
| Uncertainty | Std of top predictor | >0.1 | Doubt in leader | boundary.unc_winner |
| Flag | AND(gap, H or unc) | True → Review | Overall caution | boundary.flag |
graph LR
A["H = 0.00<br/>One Leader<br/>p=(1,0,0,0,0)"] --> B["H = 1.15<br/>Strong Lead<br/>p=(0.6,0.2,0.1,0.07,0.03)"]
B --> C["H = 1.41<br/>Review Point<br/>p=(0.4,0.25,0.2,0.1,0.05)"]
C --> D["H = 1.59<br/>Spread High<br/>p=(0.25,0.22,0.2,0.18,0.15)"]
D --> E["H = 1.61<br/>Full Spread<br/>p=(0.2,0.2,0.2,0.2,0.2)"]
style A fill:#e8f5e9,stroke:#1b5e20
style B fill:#e8f5e9,stroke:#1b5e20
style C fill:#fff3e0,stroke:#ef6c00
style D fill:#fff3e0,stroke:#ef6c00
style E fill:#fce4ec,stroke:#c2185b
Caption: Entropy progression for 5 options. Alt text: Linear graph of 5 nodes from low to max H, green low, amber medium, pink high.
mindmap
root((Common Issues))
Temperature Variation
Symptom: Entropy shifts across runs
Cause: Unfixed τ
Fix: Set τ=1.0 in configuration
Few Candidates
Symptom: Rare flags with k<3
Cause: Limited options
Fix: Normalize H / log k
Overlooked Uncertainty
Symptom: Flags on stable narrow gaps
Cause: No unc check
Fix: Require unc >0.05
Excessive Flags
Symptom: Over half routes flagged
Cause: Strict limits
Fix: Increase δ_g=0.1; track shadows
Caption: Mind map of pitfalls. Alt text: Root with four branches: temperature, candidates, uncertainty, flags, each with symptom/cause/fix.
Detailed Troubleshooting:
| Issue | Symptom | Cause | Fix | Check |
|---|---|---|---|---|
| Temperature Drift | H inconsistent | Variable τ | Fix τ=1.0 | Same U yields same H |
| Low k Bias | Infrequent flags | Few options | Normalize H/log k | Adjusted H > threshold |
| Uncertainty Ignored | Flags on low-unc | Missing gate | Add unc>0.05 | Audit sample traces |
| Over-Flagging | >50% flags | Tight δ | Raise δ_g=0.1, δ_h=1.2 | Flag rate <30% |
Limitation: Fixes assume access to code; config-only for users.
journey
title Boundary Exploration Sequence
section Test
Process Ambiguous Prompt: 5: User
Check H>1.0 in Trace: 4: User
Contrast with Clear Case: 3: User
section Tune
Edit boundary.py: 5: User
Test δ_h Variations: 4: User
Evaluate on Set: 3: User
section Extend
Incorporate KDE on U: 5: User
Monitor Boundary Changes: 4: User
Submit Pull Request: 5: User
section Apply
Update Notes: 4: User
Share with Group: 5: Group
Track in Use: 5: Group
Caption: Path from testing to application. Alt text: Journey with four sections, 11 steps, blue learning, green control.
Action Items:
- Detect a Boundary: Route "Best quadratic for [vague spec]?"; search trace for flag true. Confirm H>1.0, gap<0.05.
- Adjust Thresholds: Clone repo; change δ_h in boundary.py; run tests, note flag changes.
- Add Features: Include kernel density on U history; plot gap-H; propose pull request to diagnostics.
Limitation: Actions need repository access; start with traces for observation.
graph TD
A[Boundary Gap and Entropy] --> B[Design Notes]
A --> C[Output Format]
A --> D[Glossary]
A --> E[Source Papers]
B --> B1[Tie Resolution]
C --> C1[Boundary Details]
D --> D1[Uncertainty Terms]
E --> E1[Cortes and Vapnik 1995<br/>Support-Vector Networks]
E --> E2[Shannon 1948<br/>Mathematical Theory of Communication]
A --> F[Repository]
F --> F1[github.com/PaulTiffany/compitum]
style A fill:#e3f2fd,stroke:#01579b
style F1 fill:#e8f5e9,stroke:#1b5e20
Caption: Connections to other materials. Alt text: Graph with central node linking to docs and papers, 12 nodes, blue core, green repo.
Further Reading:
- Design-Notes: Strategies for close selections.
- Output-Format: Certificate boundary structure.
- Glossary: Definitions for shadow price, epistemic uncertainty.
- Repository: https://github.com/PaulTiffany/compitum (code and tests).
- Cortes and Vapnik (1995): Support-vector networks.
- Shannon (1948): A mathematical theory of communication.
Last Tuned: November 08, 2025 · Compitum v1.0+ · Open to Contributions