Cybernetic Editor Protocol (CEP) v2.0

Deterministic Publishing Engine using Algorithmic Supremacy

A non-LLM, deterministic document processing system that transforms manuscripts into publication-quality PDFs using formal grammars, statistical analysis, and ternary logic validation.

🎯 Core Philosophy

• NO LLMs, NO Generative AI - Pure deterministic algorithms
• Ternary Logic - Conflict (-1), Neutral (0), Harmony (+1) state system
• Mathematical Rigor - Every decision is auditable and explainable
• Self-Healing - Automatic LaTeX error detection and repair
• Fast - Process 100k word novel in under 60 seconds

✨ v2.0 Enhancements

🧠 Bayesian Learning

• Adaptive Thresholds: Learn from user corrections using Bayesian inference
• Personalized Analysis: Thresholds adapt to your writing style
• SQLite Persistence: Learning data stored in ~/.cep/learning.db
• Deterministic: Same corrections always produce same results

📐 Earley Parser with Formal CFG

• Context-Free Grammar: Replace regex with formal grammar-based parsing
• Error Recovery: Graceful handling of malformed input
• Multiple Grammars: manuscript, novel, guide, screenplay
• Parse Tree Visualization: Debug parsing with tree output

🔄 Cybernetic Feedback (T-MDP)

• Ternary Markov Decision Processes: Optimal action selection under uncertainty
• Feedback Loops: Analysis, healing, validation, learning loops
• Homeostatic Regulation: Maintain global state sum > 0
• Self-Optimization: System learns optimal policies through experience

🖥️ Mega-TUI (Terminal User Interface)

• 7 Integrated Screens: Manuscripts, Runs, Analysis, Ternary, LaTeX, Config, Ledger
• Obsidian Glass Theme: Beautiful dark theme with neon highlights
• Command Transparency: Every action shows underlying CLI command
• Real-Time Monitoring: Watch pipeline execution live
• Keyboard-First: Full navigation without mouse
• Progressive Learning: Guide novices, empower power users

Quick Launch:

# Launch TUI
cep --tui
cep-tui

# Launch with manuscript
cep-tui manuscript.txt

# Launch to specific screen
cep-tui --screen analysis

🚀 Quick Start

Installation

cd /home/ubuntu/cep_v1
pip install -e .

Basic Usage

# Process a manuscript to PDF
cep manuscript.txt -o output.pdf

# Generate statistical analysis only
cep manuscript.txt -f analysis -o analysis_report.txt

# Generate LaTeX without compiling
cep manuscript.txt -f tex -o manuscript.tex

# Process with minimal output
cep manuscript.txt -o output.pdf --quiet

Getting the Code for Testing

You can test CEP on another machine either by cloning the repository or by downloading a ZIP archive. Cloning is preferred because it preserves Git history (useful for git status, git diff, and pull-based updates) and keeps hashes that some scripts expect.

Option A: Clone the repository (recommended)

git clone https://github.com/kellyjonbrazil/cep.git
cd cep
pip install -e .[docx]
pytest

Option B: Download the ZIP

curl -L -o cep.zip https://github.com/kellyjonbrazil/cep/archive/refs/heads/main.zip
unzip cep.zip
cd cep-main
pip install -e .[docx]
pytest

The ZIP contains the same source files but omits the .git directory, so you cannot inspect commit history, use git describe, or apply incremental updates without downloading a new archive. If you only need a one-off offline test snapshot, the ZIP works; if you expect to sync changes or submit patches, use git clone.

Python API

from cep import CEPOrchestrator

orchestrator = CEPOrchestrator(verbose=True)
results = orchestrator.process_document(
    input_file='manuscript.txt',
    output_format='pdf'
)

if results['success']:
    print(f"Success! Output: {results['output_file']}")

📦 Features

1. Multi-Format Input Parsing

• Plain Text (.txt)
• Markdown (.md)
• Microsoft Word (.docx)
• LaTeX (.tex)
• PDF (.pdf)

Auto-detects format and parses into unified internal representation.

2. Statistical Analysis Engine

Deterministic, mathematically explainable metrics:

• Pacing Score - Sentence length variance (dynamic vs monotone)
• Repetition Hunter - N-gram analysis for overused phrases
• Vocabulary Diversity - Type-Token Ratio per chapter
• Readability Metrics - Flesch-Kincaid, Gunning Fog indices
• Dialogue Balance - Ratio of dialogue to narrative

3. Ternary State Machine

Based on Gated Synthesis computational philosophy:

• Assigns state (-1, 0, +1) to every paragraph
• Tracks global state sum per chapter
• Refuses to compile if global state sum ≤ 0
• Provides conflict resolution suggestions

4. Self-Healing LaTeX

Finite State Machine for error recovery:

• Parses LaTeX log files for error patterns
• Maps errors to deterministic repair functions
• Tracks repair history to avoid infinite loops
• Detailed explanations of all fixes

5. LaTeX Dependency Management

• Detects LaTeX installation (pdflatex, xelatex, lualatex)
• Verifies installation health
• Provides platform-specific installation instructions
• Handles cross-platform differences

🏗️ Architecture

Input (txt/md/docx/tex/pdf)
    ↓
[Parser] → Unified AST
    ↓
[Analyzer] → Statistical Metrics
    ↓
[State Machine] → Ternary Validation
    ↓
[LaTeX Generator] → .tex file
    ↓
[Healer] → Error Detection & Repair
    ↓
[Compiler] → PDF Output

📊 Example Analysis Output

=============================================================
STATISTICAL ANALYSIS REPORT
=============================================================

OVERALL STATISTICS
-------------------------------------------------------------
Word Count: 87,543
Sentence Count: 4,821
Avg Sentence Length: 18.2 words

PACING ANALYSIS
-------------------------------------------------------------
Variance: 42.3
Std Deviation: 6.5
Assessment: DYNAMIC

VOCABULARY DIVERSITY
-------------------------------------------------------------
Type-Token Ratio: 0.487
Unique Words: 8,234
Assessment: MODERATE_DIVERSITY

READABILITY METRICS
-------------------------------------------------------------
Flesch-Kincaid Grade: 8.7
Gunning Fog Index: 10.2
Grade Level: HIGH_SCHOOL

🔒 Ternary State Validation

[Validating Chapter: The Beginning]
  [+1] chapter_1_para_0: Passed all 5 validation checks (confidence: 1.00)
  [0] chapter_1_para_1: Passed 3/5 validation checks (confidence: 0.60)
  [-1] chapter_1_para_2: Failed 4/5 validation checks (confidence: 0.80)
  
Chapter State Sum: 0

⚠ COMPILATION BLOCKED: Global state sum ≤ 0
Conflict Resolution Required:

1. chapter_1_para_2
   → Escape special LaTeX characters (#, $, %, &)
   → Balance quotation marks

🛠️ Requirements

Python Dependencies

• Python 3.8+
• Standard library only for core functionality
• Optional: python-docx for .docx support
• Optional: pypdf for .pdf support

LaTeX (for PDF compilation)

• Linux: sudo apt-get install texlive-full
• macOS: Install MacTeX
• Windows: Install MiKTeX or TeX Live

📖 Documentation

• ARCHITECTURE.md - System design and data flow
• WHY_NOT_AI.md - Philosophy and rationale
• API_REFERENCE.md - Complete API documentation
• TUI_GUIDE.md - Mega-TUI user guide
• TUI_ARCHITECTURE.md - TUI technical architecture
• TUI_KEYBINDINGS.md - Complete keybinding reference

🧪 Testing

# Run test suite
python -m pytest tests/

# Test with sample manuscript
cep samples/sample_manuscript.txt -o test_output.pdf

🎓 Philosophy

CEP embodies the Gated Synthesis computational philosophy:

1. Ternary States - Every component can be in Conflict, Neutral, or Harmony
2. Gated Operations - Current state determines which operations are permitted
3. Bayesian Learning - Update beliefs based on observed outcomes
4. Graceful Degradation - Conflict state produces safe fallback, not crashes
5. Auditability - Every decision is mathematically explainable

Why not LLMs?

• ✅ Deterministic - Same input → Same output, always
• ✅ Fast - No API calls, no GPU required
• ✅ Auditable - Clear logic, no black boxes
• ✅ Reliable - No hallucinations, no token limits
• ✅ Offline - Works without internet

🤝 Contributing

CEP is designed for clarity and maintainability:

1. All functions have docstrings
2. Type hints throughout
3. Modular architecture
4. Comprehensive test coverage

📜 License

Open Source - See LICENSE file for details

🎯 Roadmap

• Core parsing engine
• Statistical analysis
• Ternary state machine
• Self-healing LaTeX
• CLI interface
• Bayesian learning from corrections
• Cumulative mastery ledger
• Earley parser implementation
• Gaussian confidence tracking
• Web interface

💡 Inspiration

Built on the shoulders of giants:

• Gated Synthesis - Computational philosophy framework
• Perfect Parser Initiative - Formal grammar design
• Sentinal Framework - Cybernetic principles
• Classical Parsing Theory - Earley, CYK, recursive descent

---

Version: 1.0.0
Status: Production Ready
Philosophy: Code over conversation, algorithms over approximation