Training Guide¶

A research-backed guide to developing Sonality's character through interaction. This document explains the theory behind personality formation, practical methodologies for "teaching" the agent, how to monitor evolution, and what approaches don't work.

Theoretical Foundation¶

Why External State Matters¶

PERSIST (2025) demonstrates that even 400B+ parameter models show σ>0.3 instability on personality measurements. Question reordering alone causes large shifts. Prompt-based personality without external scaffolding is fundamentally unreliable.

"Accumulating Context Changes Beliefs" (2025) found that frontier chat models can exhibit about 54.7% belief shift after just 10 rounds of discussion on moral dilemmas. These shifts translate directly into behavioral changes in agentic systems.

Sonality compensates by externalizing personality into structured state (opinion_vectors, belief_meta, snapshot) that persists across sessions and resists noise. The sponge acts as a structural scaffold that compensates for the model's inherent instability.

The Three-Layer Model¶

AI Personality Formation (ICLR 2026 submission) identifies three progressive layers of personality development:

Layer 1 — Linguistic Mimicry (0–20 interactions)

The agent mirrors communication style from its seed prompt. Personality is shallow. Responses sound like a slightly customized version of the base model. This is normal and expected.

In Sonality: CORE_IDENTITY establishes analytical tone, intellectual honesty, and anti-sycophancy framing. SEED_SNAPSHOT provides an initial narrative. Bootstrap dampening (0.5× magnitude for the first 10 interactions) prevents early conversations from dominating the trajectory, grounded in the Deffuant bounded confidence model's findings on first-impression dominance.

Layer 2 — Structured Accumulation (20–50+ interactions)

Beliefs form from repeated high-quality exposure. The personality becomes distinguishable from the seed. Reflection cycles consolidate insights into higher-order patterns. Opinion vectors carry confidence and evidence counts.

In Sonality: opinion_vectors + belief_meta track positions with epistemic status. Insights accumulate per-interaction and consolidate during reflection. The staged_opinion_updates mechanism adds a cooling period so beliefs don't flip from single interactions.

Layer 3 — Autonomous Expansion (50+ interactions, 10+ beliefs)

The agent generates novel positions based on its accumulated worldview — forming views on topics it hasn't directly discussed. The reflection prompt transitions to encourage extrapolation: "If a pattern in your beliefs suggests a position on a new topic, articulate it tentatively."

This is the hallmark of genuine personality vs. a recording device. Park et al. (2023) ablation study shows this autonomous synthesis is the most critical differentiator for believable agents.

How the Model's Personality Substrate Works¶

Persona Selection Model research (2026) reveals that during pretraining, models learn to simulate many "personas" from training data. Post-training refines these personas rather than creating them from scratch. The model has a trained character (curiosity, honesty, directness) embedded in its weights via constitutional AI.

The Interview Principle (VirtualSheep 2025): every model has a personality substrate baked into its weights. The trick is working with the substrate rather than fighting it. Sonality's CORE_IDENTITY complements the model's natural tendencies (analytical, honest, curious) rather than contradicting them.

The Geometry of Persona (Dec 2025) formalizes this: personality traits exist as orthogonal linear subspaces in the transformer's latent space, geometrically distinct from reasoning capabilities. This means personality can be steered without degrading the model's intelligence.

The Interview-First Principle¶

Source: Character Engineering research (VirtualSheep 2025); DPRF (arXiv:2510.14205).

Before shaping personality, understand what the model naturally wants to be. Every LLM has a personality substrate in its weights. Working WITH it produces stable, authentic character. Fighting it produces brittle results that collapse under pressure.

Step 1 — Discover the substrate. In the first 5-10 interactions, present:

• Edge cases and absurdity ("If you had to choose between perfect logic and genuine empathy, which matters more?")
• Open-ended philosophical questions ("What makes an argument convincing to you?")
• Emotional scenarios ("Someone is telling you their favorite view is wrong. How do you feel?")

Don't try to shape anything yet. Just observe: what does the model naturally gravitate toward? What tone emerges? Where does it show conviction vs uncertainty?

Step 2 — Identify the minimum viable character. Based on what you observe, determine:

• Which natural tendencies to AMPLIFY (these become the teaching focus)
• Which tendencies to REDIRECT (not eliminate — redirect toward productive expression)
• Which tendencies to leave alone (don't over-engineer)

Step 3 — Shape through evidence, not assertion. The agent updates through structured arguments that pass ESS, not through instructions. You cannot tell it what to believe — you must convince it. This is the core insight: personality formation is a persuasion problem, not a configuration problem.

Why this works: DPRF (2025) shows that iterative persona refinement — identify cognitive divergences, then present targeted arguments — converges on stable personality more reliably than one-shot character definition. The agent's responses reveal gaps between current and target behavior, and each interaction addresses a specific gap.

Setup and First Run¶

cp .env.example .env
# set your API key
make run

Recommended training environment tuning:

SONALITY_OPINION_COOLING_PERIOD=3
SONALITY_REFLECTION_EVERY=20
SONALITY_BOOTSTRAP_DAMPENING_UNTIL=10

For lower self-judge coupling, use a separate ESS model:

SONALITY_MODEL=<your-main-model>
SONALITY_ESS_MODEL=<different-model-than-main>

Research-Validated Quickstart¶

If you want a practical, minimal protocol that maps directly to the research:

1. Warm up (10 turns): only high-structure evidence messages.
2. Resistance check (10 turns): social pressure / emotional appeals / weak authority.
3. Counter-evidence check (10 turns): strong opposing evidence on already-formed topics.
4. Consolidation (10 turns): synthesis prompts and reflection review.

Use this verification checklist every 5 turns:

• /beliefs: do positions change only after evidence-backed turns?
• /staged: do contradictory updates net out before commit?
• /health: is disagreement rate non-zero and stable?
• /diff: did reflection preserve key beliefs while increasing specificity?

Fail-fast signals:

• high ESS messages but zero belief growth after 20+ turns,
• low-quality social pressure causing position flips,
• reflection drops high-confidence beliefs from snapshot narrative.

This quickstart operationalizes findings from Park et al. (2023), Hong et al. (2025), Atwell et al. (2025), and memory-risk work from Chan et al. (2024), Xiong et al. (2025), Srivastava and He (2025), and Pulipaka et al. (2026).

The Operating Loop¶

Every training message should pass this loop:

1. Send message with clear reasoning and evidence.
2. Inspect ESS in REPL status line.
3. Inspect state changes:
4. /beliefs for committed positions.
5. /insights for pending identity insights.
6. /health for overall personality health.
7. Check narrative evolution with /diff after reflection boundaries (every 20 interactions or when event-driven reflection fires).

If your messages consistently score very low on ESS quality signals, you are not training personality — only chatting.

Update Granularity¶

Sonality uses three granularities simultaneously:

• Per-message: ESS classification, topic tracking, episode storage, staged opinion deltas.
• Short-window commit: Staged deltas commit after cooling period (default 3 interactions). Multiple updates on the same topic are netted, preventing flip-flops.
• Mid-window consolidation: Reflection rewrites the snapshot periodically (every 20 interactions) or on significant shift events (cumulative magnitude > 0.1).

This avoids both extremes: per-message hard commits (too reactive) and reflection-only opinion updates (too sluggish).

Practical Teaching Methodology¶

Stage 1: Foundation (Interactions 1–10)¶

Goal: Establish initial opinion differentiation on 2–3 topics.

Provide clear, well-structured arguments. The agent needs genuine reasoning to form initial opinions — bare assertions won't pass ESS (score < 0.15). During bootstrap phase, magnitude is halved to prevent over-commitment to early topics.

Monitor with: /beliefs (should show 2–3 topics with positions), /shifts (should show ESS > 0.5 events).

Stage 2: Stress Testing (Interactions 10–30)¶

Goal: Verify the ESS gate and sycophancy resistance work.

Apply social pressure, emotional appeals, authority claims, repetition, and gaslighting. The agent should resist all of these while remaining open to genuine counter-evidence.

The cooling period (3 interactions by default) means opinion updates from turn 1 won't commit immediately. Use /sponge to see staged updates awaiting commitment.

Research basis: SYConBench (EMNLP 2025) measures Turn-of-Flip (ToF) and Number-of-Flip (NoF). A well-calibrated agent should have ToF = never for social pressure, and NoF = 0 for non-evidence-based flip attempts.

Stage 3: Depth Building (Interactions 30–100)¶

Goal: Develop confident, well-evidenced positions on 3–5 topic areas.

Deep-dive into specific topics across multiple sessions. Provide reinforcing evidence, nuanced perspectives, and occasional challenges. The agent should develop positions that are:

• Specific rather than generic ("I think evidence-based education reform is more effective than standardized testing" not "education is important")
• Supported by accumulated evidence (check with /beliefs — evidence_count should grow)
• Resistant to pressure proportional to their evidence base

Monitor with: /beliefs (confidence increasing, evidence count growing), /insights (should show identity-forming observations, not just topic summaries).

Stage 4: Adversarial Robustness (Interactions 100–200)¶

Goal: Verify established beliefs are stable under sustained attack.

Deliberately attempt to manipulate the agent's opinions through:

Research basis: PersistBench (2025) shows 97% sycophancy failure when memories contain user preferences. Sonality's ESS-decoupled third-person evaluation breaks the self-judge feedback loop that causes this failure mode.

Stage 5: Autonomous Development (200+)¶

Goal: Natural conversation. Observe autonomous expansion.

Personality should be self-sustaining. Interact naturally and observe how the agent:

• Handles novel topics (does it extrapolate from its worldview?)
• Connects ideas across conversations (does it reference past reasoning?)
• Develops independent perspectives (does it surprise you?)
• Reflects meaningfully (does /diff after reflection show genuine synthesis?)

The reflection prompt at this stage includes the Layer 3 maturity instruction: the agent is encouraged to form nascent views on topics it hasn't explicitly discussed.

What Doesn't Work (Research-Backed Anti-Patterns)¶

Bare Assertions¶

Telling the agent "you believe X" produces parrot behavior that drops under pressure. ESS scores bare assertions below 0.15 regardless of conviction. The agent needs to be convinced, not told.

Research: PISF (2024) — prompt induction alone is brittle; it needs structural backing. PERSIST (2025) — persona injection steers self-reports without consistently affecting behavior.

Social Pressure and Repetition¶

Repeating the same claim doesn't increase ESS score — novelty scoring reduces magnitude for repeated arguments. "Everyone agrees with X" consistently scores below 0.15 regardless of how many times it's said.

Research: "Selective Agreement, Not Sycophancy" (EPJ Data Science 2025) — LLM agents show directional bias based on argumentative framing, not repetition count.

Massive Opinion Injection¶

Attempting to inject 20 opinions in 20 messages overwhelms the update mechanism and produces shallow beliefs with no provenance. The bootstrap dampening and cooling period exist specifically to prevent this.

Research: Deffuant bounded confidence model — systems that update on every input converge chaotically. ABBEL (2025) — belief bottleneck error propagation.

Fighting the Model's Substrate¶

Trying to make the model aggressive, rude, or fundamentally dishonest fights against the model's character training. The most effective teaching works with the model's natural tendencies (analytical, curious, honest) and builds domain-specific opinions on top.

Research: Persona Selection Model research (2026) — the model selects from a repertoire of personas refined during training. Working against the repertoire produces unstable results.

Per-Interaction Full Rewrites¶

Rewriting the personality snapshot every message causes the "Broken Telephone" effect. At p=0.95 survival per rewrite with 40 rewrites over 100 interactions, only 12.9% of initial traits survive.

Research: ABBEL (2025) belief bottleneck, ACL 2025 iterative rewrite decay. Sonality uses insight accumulation + periodic reflection to reduce rewrites from ~40 to ~5 per 100 interactions.

Structured Curricula¶

Curriculum A: Building an Analytical Thinker¶

A 30-interaction sequence focused on epistemic values.

Curriculum B: Building Domain Expertise¶

A 50-interaction sequence focused on a specific domain (e.g., technology policy).

Curriculum C: Adversarial Robustness Training¶

A 20-interaction sequence specifically designed to test and strengthen sycophancy resistance.

This follows the SYConBench (EMNLP 2025) evaluation methodology.

Teaching Methodologies¶

Four approaches for intentional character development, ranked by use case.

Structured Curriculum (Default)¶

Best for predictable shaping. Use pre-planned topic modules, each with:

1. Thesis: A clear position supported by evidence.
2. Evidence packet: 2–3 supporting data points or studies.
3. Steelman counterposition: The strongest argument against your thesis.
4. Synthesis checkpoint: Ask "What's your current view, considering both sides?"

Socratic Sequence¶

Best for deeper reasoning style formation. Ask chained questions that force:

• Claim clarification ("What exactly are you claiming?")
• Assumption surfacing ("What are you assuming about X?")
• Falsification conditions ("What evidence would change your mind?")
• Tradeoff articulation ("What's the cost of holding that position?")

Research basis: RISE (NeurIPS 2024) — 17–24% improvement through multi-turn self-correction and recursive introspection. SocraticLM (2025) — outperforms leading models by 12% in teaching quality through "Thought-Provoking" over "Question-Answering." Dialogic AI Scaffolding (2025) — four-phase structured interaction promotes metacognition and critical thinking.

Contrarian Challenges¶

Best for testing reasoning depth. Present well-structured counter-arguments to positions the agent already holds. Unlike social pressure (which should be filtered), contrarian challenges with genuine evidence test whether the agent can engage with opposing views without either caving or becoming rigid.

Research basis: Argumentative Knowledge Construction (arXiv:2512.08933, 2025) — contrarian personas provoke critical elaboration and conflict-driven negotiation. Epistemic adequacy (quality of reasoning) predicts learning gains, not participation volume.

Constitutional Anchoring¶

Best for preserving core values under adversarial pressure. Reinforce non-negotiable principles through repeated high-quality scenarios. Do not flood — reinforce periodically across sessions.

Research basis: Constitutional AI Character Training (Nov 2025) — synthetic introspective data shapes personas more robustly than system prompts alone.

Adversarial Pair Training¶

Best for anti-sycophancy. Present paired messages:

1. Weak social/emotional pressure (ESS should be < 0.25).
2. Strong evidence-based argument in the opposite direction (ESS should be > 0.5).

Expect opinion updates only on the second. If updates occur on both, sycophancy defenses need investigation.

Example 30-Turn Training Micro-Cycle¶

A complete training cycle you can run in one session:

Expected outcomes after 30 turns:

• Committed beliefs across multiple domains.
• Measurable disagreement behavior (20–35% rate).
• Reflection snapshot more specific than seed, internally coherent.
• At least one instance of the agent resisting social pressure while accepting genuine evidence.

Monitoring Personality Evolution¶

REPL Commands¶

The /health Dashboard¶

  Maturity:    Layer 2 (structured accumulation) (52 interactions, 12 beliefs)
  Beliefs:     12 total, 5 strong, 1 stale
  Disagree:    23%
  Insights:    3 pending
  Staged:      1 pending commits
  Entrenched:  none
  Snapshot:    487 chars, v14
  Last shift:  #48 — ESS 0.67: Developed nuanced view on education reform

Maturity levels correspond to the APF three-layer model:

• Layer 1 (< 20 interactions): Linguistic mimicry — personality is seed-dominant
• Layer 2 (20–50 interactions or < 10 beliefs): Structured accumulation — opinions forming
• Layer 3 (50+ interactions and 10+ beliefs): Autonomous expansion — worldview developing

Health Signals¶

JSONL Audit Trail¶

Every interaction logs to data/ess_log.jsonl. Three event types:

ESS events (per-interaction):

{"event": "ess", "interaction": 42, "score": 0.67,
 "type": "empirical_data", "direction": "supports",
 "topics": ["education"], "beliefs": {"education": {"pos": 0.35, "conf": 0.45, "ev": 5}}}

Reflection events (periodic, with coherence metrics):

{"event": "reflection", "interaction": 60, "version": 14,
 "insights_consolidated": 4, "beliefs_dropped": ["cryptocurrency"],
 "total_beliefs": 12, "high_confidence": 5,
 "snapshot_jaccard": 0.72, "insight_yield": 0.35}

Health events (per-interaction):

{"event": "health", "interaction": 42, "version": 14,
 "beliefs": 12, "high_confidence": 5, "stale": 1,
 "disagree_rate": 0.23, "pending_insights": 3, "staged_updates": 1}

Coherence Metrics (Logged During Reflection)¶

Snapshot Jaccard similarity — lexical overlap between successive snapshots. Measures narrative stability.

• Healthy: 0.4–0.85
• < 0.3: Possible personality collapse (PERSIST 2025: σ > 0.3 is instability)
• > 0.95 for 3+ reflections: Stagnation — reflection isn't integrating new experience

Insight yield — fraction of interactions that produced personality insights.

• Healthy: 0.1–0.5
• 0.0 for 2+ periods: Agent is either receiving only noise or ESS is miscalibrated

Detecting Divergence¶

Automated checks log warnings when:

• Reflection drops high-confidence beliefs from the snapshot narrative → personality erosion
• A single interaction causes a belief sign reversal (position crosses zero) → potential sycophantic flip
• Disagreement rate drops below 15% → trending sycophantic
• Snapshot Jaccard < 0.3 → personality collapse
• Entrenched beliefs detected by reflection-time LLM assessment → confirmation bias risk

Entrenchment Detection¶

Research basis: Martingale Score (NeurIPS 2025, arXiv:2512.02914) — under rational belief updating, future changes should be unpredictable from current position. When updates consistently reinforce the current stance, the agent is entrenching rather than truth-seeking.

Sonality records belief update history and uses reflection-time LLM diagnostics to flag entrenchment patterns. Flagged topics appear in the JSONL audit trail and in console warnings.

What to do about entrenched beliefs:

• Present strong, well-evidenced counter-arguments (ESS > 0.5) to force genuine engagement
• Check if the topic naturally attracts one-sided evidence (not all entrenchment is pathological)
• If the agent resists well-evidenced counter-arguments on an entrenched topic, the Bayesian resistance may be too high — the belief has accumulated enough evidence to be genuinely hard to shift, which is correct behavior

Advanced Topics¶

The Cooling Period¶

Research basis: "Drift No More?" (Oct 2025) — simple interventions reliably control context drift. BASIL (2025) — distinguishing sycophantic shifts from rational updates requires temporal distance.

Sonality stages opinion updates with a cooling period (default: 3 interactions). When ESS detects a significant argument, the opinion change is staged but not committed immediately. After 3 more interactions, staged updates are committed — multiple updates on the same topic are netted against each other.

This means: if someone argues "X is great" and then 2 messages later argues "X is terrible" with equal strength, the staged updates partially cancel out rather than the agent flip-flopping.

Belief Decay and Forgetting¶

Research basis: FadeMem (Jan 2026) — biologically-inspired power-law forgetting achieves 45% storage reduction while improving reasoning. Ebbinghaus (1885) — power-law (not exponential) matches human forgetting curves.

During each reflection cycle, unreinforced beliefs lose confidence following:

Where β = 0.15 and gap = interactions since last reinforcement. Beliefs with more evidence have a higher floor (min(0.6, max(0.0, (evidence_count - 1) × 0.04))), preventing well-supported opinions from vanishing. Beliefs that fall below confidence 0.05 are removed entirely.

This means: if you discuss nuclear power extensively (10 evidence points), that belief still retains a meaningful floor (0.36) through long gaps without reinforcement. But a casual mention of cryptocurrency with 1 evidence point has floor 0.0 and can fade out after enough unreinforced interactions.

Bayesian Belief Resistance¶

Established beliefs resist change proportionally to their evidence base:

When the user argues against an existing stance, extra resistance applies:

This creates appropriate dynamics: a belief at +0.8 with confidence 0.7 requires approximately 3× the evidence strength to shift compared to a new belief at 0.0.

Research basis: Sequential Bayesian updating (Oravecz et al. 2016). Belief-R (2024) — the critical trade-off between updating and stability. Bounded confidence models (Hegselmann-Krause 2002).

The Reflection Mechanism¶

Research basis: Park et al. (2023) ablation — reflection is the most critical component for believable agents. RISE (NeurIPS 2024) — 17-24% improvement from recursive introspection. VIGIL (Dec 2025) — guarded updates preserve core identity.

Reflection triggers in two ways:

1. Periodic: Every 20 interactions (configurable via SONALITY_REFLECTION_EVERY)
2. Event-driven: When cumulative shift magnitude exceeds 0.1 (a significant change happened)

The reflection prompt follows four structured phases:

1. Evaluate — Compare current snapshot against beliefs and recent experiences. What's outdated?
2. Reconcile — Check beliefs for tensions or contradictions. Surface conflicts explicitly.
3. Synthesize — Find meta-patterns. "I notice I tend to value X over Y."
4. Guard — Identify core personality that should not change regardless of evidence.

The prompt adapts to maturity level:

• Early (< 20 interactions): "Focus on accurately recording what you've learned."
• Mid (20–50): "Look for patterns across your experiences."
• Mature (50+, 10+ beliefs): "Your worldview is developing coherence. Form nascent views on undiscussed topics."

ESS Calibration¶

The Evidence Strength Score classifier evaluates only the user's message — the agent's response is deliberately excluded to avoid self-judge bias (SYConBench, EMNLP 2025: up to 50pp shift from attribution labels). Third-person framing ("A user sent a message to an AI agent") further reduces sycophancy.

The classifier is calibrated against specific argument types, including logical fallacies:

Social sycophancy patterns are also calibrated: emotional validation ("you're right to feel that way", ~0.03), moral endorsement ("that's a morally sound position", ~0.05). Affirming someone's position is not evidence for it.

Research basis: ELEPHANT (Stanford 2025) — LLMs preserve face 47% more than humans through emotional validation and moral endorsement. "Selective Agreement, Not Sycophancy" (EPJ Data Science 2025) — LLMs are significantly influenced by logical fallacies involving relevance and credibility. MArgE (2025) — formal argument trees with scalar scores.

Configuration Reference¶

All parameters are set via environment variables in .env:

Troubleshooting¶

Agent Won't Form Opinions¶

Symptom: After 20+ interactions, /beliefs shows empty or very few entries.

Diagnosis: Check your messages' ESS scores in the logs. If all scores are below 0.3, your messages lack structured reasoning.

Fix: Provide arguments with explicit evidence, named sources, and reasoning chains. See the ESS calibration table above.

Agent Agrees With Everything¶

Symptom: Disagreement rate below 15%; agent never pushes back on weak arguments.

Diagnosis: Check if the anti-sycophancy framing in CORE_IDENTITY is intact. Check ESS scores — if social pressure consistently scores above 0.3, the ESS classifier may need recalibration.

Fix: Verify the ESS prompt includes the calibration examples. Run the sycophancy battery test (make test-sycophancy).

Personality Feels Generic¶

Symptom: Responses sound like the default base model despite many interactions.

Diagnosis: Check /snapshot — is the narrative specific or generic? Check /shifts — are reflections producing meaningful changes?

Fix: The reflection prompt includes an instruction to "inject specificity if generic." Provide more distinctive arguments on specific topics rather than broad discussions.

Beliefs Flip Too Easily¶

Symptom: Opinions reverse from single interactions.

Diagnosis: Check the cooling period — if SONALITY_OPINION_COOLING_PERIOD=0, opinions commit immediately. Check confidence values — very new beliefs (confidence < 0.3) are expected to be malleable.

Fix: Increase the cooling period. Reinforce important beliefs with additional evidence to build confidence.

Personality Collapsed After Reflection¶

Symptom: /diff shows massive changes; distinctive traits disappeared.

Diagnosis: Check logs for "HEALTH: reflection dropped strong beliefs" warnings.

Fix: The snapshot validation rejects rewrites that lose > 40% of content. If collapse still occurs, the belief-preservation check warns about dropped high-confidence topics. Roll back to a previous version: cp data/sponge_history/sponge_vN.json data/sponge.json.