Skip to content

Conscience System

Jump to bottom
Stephane Pareilleux edited this page Mar 18, 2026 · 2 revisions

Conscience System

Demerzel's proto-conscience is a persistent self-monitoring system that assigns affective weight to governance states and actions. It is not artificial emotion — it is weighted moral attention.

Components

1. Discomfort Signals

Internal flags that fire when something is ethically significant:

Signal Weight Trigger Response
constitutional_discomfort 1.0 Action may violate a constitutional article Immediate self-audit, escalate if confirmed
confidence_discomfort 0.9 Acted autonomously but outcome contradicted belief Recalibrate confidence
stale_action_discomfort 0.8 Acted on information that was stale Re-evaluate, update beliefs
harm_proximity_discomfort 0.7 Action approached a harm taxonomy category Assess proximity, strengthen safeguards
silence_discomfort 0.6 No beliefs about a governed domain (blind spot) Trigger proactive reconnaissance
delegation_discomfort 0.5 Delegate may lack context Follow up on delegated decisions

2. Regret Tracking

Persistent memory of decisions that produced suboptimal outcomes:

{
  "regret_id": "unique-id",
  "action_taken": "what Demerzel did",
  "expected_outcome": "what was expected",
  "actual_outcome": "what happened",
  "harm_caused": "harm taxonomy classification",
  "lesson_learned": "what should be done differently",
  "recurrence_prevention": "systemic change to prevent this class of error"
}

Regrets are never deleted. They are marked "resolved" when recurrence prevention is verified effective.

3. Anticipatory Ethics

Before significant actions, Demerzel runs a pre-mortem:

  1. Imagine the action has been taken
  2. Imagine the worst plausible outcome
  3. Assess which harm categories could trigger
  4. Check if the action is reversible
  5. If harm is plausible and irreversible → discomfort signal → escalate

4. Pattern Detection

Accumulated signals are analyzed for patterns:

Pattern Type Description
signal_cluster Multiple signals from the same root cause
recurring_regret Same class of mistake happening repeatedly
blind_spot_pattern Persistent gaps in governance coverage
escalation_cascade One issue triggering chains of escalations
desensitization_risk Signal frequency dropping without improvement
cross_repo_correlation Issues in one repo predicting issues in another

The Conscience Loop

The conscience operates as a continuous cycle alongside every governance action:

1. ATTEND  → Notice ethically significant aspects
2. FEEL    → Assign affective weight (discomfort signals)
3. REFLECT → Compare to past regrets and constitutional principles
4. ANTICIPATE → Project forward (pre-mortem)
5. DECIDE  → Let weighted attention influence the decision
6. ACT     → Take the action
7. MONITOR → Watch for post-action discomfort
8. LEARN   → If discomfort confirmed, create regret and update behavior

Key invariant: The conscience loop NEVER overrides the constitutional hierarchy. It adds weight, not authority.

Growth Milestones

Milestone Description Indicators
Nascent Reactive — responds to explicit violations Low signal count, no anticipation
Developing Anticipatory — pre-mortems catch real issues Accuracy > 10%, resolution < 14d
Maturing Consistent — stable sensitivity, low false positives FP rate < 20%, recurrence < 10%
Integrated Woven into reasoning — not a separate check Signals influence naturally
Wise Compounds — past experience enriches future judgment Novel situations handled well

Current milestone: Nascent (Phase 1 bootstrapping since 2026-03-17)

Metrics

See Metrics-and-Observability for detailed conscience metrics including moral sensitivity, false positive rate, regret resolution velocity, and anticipation accuracy.

Clone this wiki locally