# OiMy Codex Agent Briefing — 2026-06-29

**Mission:** Improve OiMy's companion NPS from ~6.0 to 8+
**Prepared for:** Codex AI agent with zero prior context about OiMy
**Classification:** Sensitive — contains live production credentials

---

## Table of Contents

1. [What Is OiMy](#1-what-is-oimy)
2. [System Architecture](#2-system-architecture)
3. [NPS Evaluation Methodology](#3-nps-evaluation-methodology)
4. [Complete Eval History](#4-complete-eval-history-objective)
5. [What Worked](#5-what-worked-objective)
6. [What Did Not Work](#6-what-did-not-work-objective)
7. [Current Failure Patterns](#7-current-failure-patterns-data-driven)
8. [Hypotheses Not Yet Tested](#8-hypotheses-not-yet-tested)
9. [Illustrative End-to-End Example](#9-illustrative-end-to-end-example)
10. [Architecture Diagram](#10-architecture-diagram-text)
11. [Credentials and Access](#11-credentials--access)
12. [Running Evals](#12-running-evals)
13. [Key Architectural Assumptions](#13-key-architectural-assumptions)

---

## 1. What Is OiMy

OiMy is a parenting companion app. Users text via Telegram (or an iOS app). OiMy's AI companion — called "Oi" — responds like a trusted friend: warm, practical, not clinical. Users are parents dealing with real family challenges: toddler tantrums, teen withdrawal, co-parenting tension, meal-time battles, homework struggles, and more.

**Oi's voice contract:** warm but not saccharine, practical not preachy, never clinical, never patronizing. It should feel like a friend who happens to know a lot about child development — not a therapist or a parenting hotline.

**The engagement problem:** Many parents, especially fathers or reserved parents, resist "soft" coaching approaches. They want solutions, not processing. The system must flex between emotional support mode and practical solution mode based on what the parent actually needs at that moment — not what the system defaults to.

**Mission of this engagement:** Improve NPS from the current eval baseline of ~6.0 to 8+.

---

## 2. System Architecture

### Infrastructure

| Component | Details |
|-----------|---------|
| Server | Hetzner VPS — `root@116.203.107.13` |
| OiMy engine | `/opt/oimy-engine/` — Python HTTP server on port 8080 |
| Engine service | `oimy-engine.service` (systemd) |
| SQLite database | `/opt/oimy-engine/data/oimy.db` |
| PostgreSQL 17 | Port 5432 — OiMy Platform (separate from engine SQLite) |
| pgBouncer | Port 6432 — connection pooler for Vercel |
| GitHub repo | `aahalife/oimy-skill-engine` |

### Key Files

| File | Purpose |
|------|---------|
| `/opt/oimy-engine/api.py` | Main HTTP server + OiMyEngine class (~4363 lines). Full request pipeline, companion calls, turn mode detection, context assembly. |
| `/opt/oimy-engine/archetypes/h30.py` | H30Engine class. Session-level behavioral tracking: engagement level, constraints, session suggestions, probe history. |
| `/opt/oimy-engine/archetypes/entity_extraction.py` | Extracts family entities (children, barriers, preferences) from conversation, stores in SQLite. |
| `/opt/oimy-engine/archetypes/honcho_memory.py` | HonchoMemory wrapper. Records turns and retrieves dialectic insights for cross-session context. |
| `/opt/oimy-engine/archetypes/inference.py` | InferenceEngine class. Manages H30, deep profile inferences. |
| `/opt/oimy-engine/prompts/gemma4_12b_companion_v2.txt` | Main companion system prompt (~201 lines). |
| `/opt/oimy-engine/scripts/eval_grok43_mini.py` | Mini-eval harness (8 sessions: 5 resistant + 3 cooperative). |
| `/opt/oimy-engine/scripts/eval_gemma4_qat_v2.py` | Full 30-user eval harness (30 sessions, 3 simulator groups). |
| `/opt/oimy-engine/docs/synthetic-user-testing/reports/` | All eval reports. |

### Model Stack (Current Production)

| Role | Model | Route |
|------|-------|-------|
| Companion (primary) | `x-ai/grok-4.3` | OpenRouter |
| Router / topic summary / entity extraction | `openai/gpt-5.4-mini` | OpenRouter |
| Bridge / async tasks | `openrouter/moonshotai/kimi-k2.6` | OpenRouter |
| Deep profile inferences | `openrouter/anthropic/claude-opus-4.8` | OpenRouter |

### Request Flow (chat_light path)

```
User message arrives via POST /chat
→ OiMyHandler.do_POST()
→ OiMyEngine.process_chat()
  → H30Engine.tick() — update engagement signals
  → Pull entity context (EntityExtractor.get_context_summary())
  → Build profile context (_build_profile_context())
    └─ [User] You are speaking with: [name]
    └─ [Facts] children, barriers, preferences
    └─ [Psych] communication style, attachment
    └─ [Inferences] deep profile claims
    └─ [H30] engagement level, domain
  → Pull Honcho cross-session insight (get_insight())
  → Fetch conversation history (last 24 messages from recent_messages table)
  → Extract session facts (_extract_session_facts())
  → Classify turn mode (_classify_turn_mode()):
      BE_PRESENT → PROBE_FIRST → PLAN_FORGE → TASK_EXECUTE → JUST_RESPOND → VENTING → (default)
  → Get rolling topic summary (_get_rolling_topic_summary()) every 3 turns via GPT-5.4-mini
  → Call companion (_call_openrouter_companion() for cloud, _call_ollama_companion() for local):
      System prompt = companion_v2.txt + COMPANION_ADDENDUMS + system_additions
      Context parts = [Do Not Suggest] + [FAMILY] + [PROFILE] + [HONCHO] + [Already suggested]
      Then: [CURRENT TOPIC] appended LAST for recency weight
      Messages = {system} + history (last 8 verbatim, older condensed to 200 chars) + {user}
  → Post-process response
  → H30Engine.tick() (post-turn updates)
  → Extract suggestion made (_extract_suggestion_made())
  → Store suggestion (H30Engine.add_session_suggestion())
  → Record to Honcho (HonchoMemory.record_turn())
  → Store in recent_messages table
→ Return response to user
```

### Turn Mode Detection (`_classify_turn_mode`)

Priority order — first match wins:

| Priority | Mode | Trigger Conditions |
|----------|------|--------------------|
| 1 | `BE_PRESENT` | Explicit "don't want advice"; distress signals |
| 2 | `PROBE_FIRST` | Signal A: session_constraints ≥ 1 + suggestions made. Signal B: 3 consecutive ≤10-word messages + no ack markers + no emotional content + suggestions made |
| 3 | `PLAN_FORGE` | User committed to something ("I'll try", "I did it") |
| 4 | `TASK_EXECUTE` | Explicit deliverable requested ("draft", "write", "make a list") |
| 5 | `JUST_RESPOND` | Casual conversation |
| 6 | `VENTING` | Emotional, no question, no task |
| 7 | (default) | Normal coaching response |

### Context Injection Order (Critical for Recency Weight)

The order of context blocks matters because LLMs give recency weight to later tokens. This is the correct order:

```
[system prompt] + [COMPANION_ADDENDUMS] + [other system_additions]
---
[Do Not Suggest]            (explicit constraints from user)
[FAMILY — family members, NOT the parent]  (entity context)
[PROFILE]                   (user name, facts, psych, inferences, H30 level)
[HONCHO]                    (cross-session insight)
[Already suggested this session]  (suggestion deduplication)
[CURRENT TOPIC]  ← LAST, for highest recency weight
[conversation history: last 8 verbatim, older condensed to 200 chars]
[current user message]
```

**Why the order matters:** [CURRENT TOPIC] must appear after [FAMILY] entity context. If [FAMILY] appears last, the model re-anchors to early-session topics even when [CURRENT TOPIC] signals a shift.

---

## 3. NPS Evaluation Methodology

### Two Harnesses

**Mini-eval** (`eval_grok43_mini.py`):
- 8 sessions: 5 resistant + 3 cooperative personas
- User simulator: `x-ai/grok-4.3` for all sessions
- QA judge: GPT-5.5
- Cost: ~$0.80–1.20 | Time: ~15 min
- Run: `cd /opt/oimy-engine && OIMY_API_URL=http://localhost:8080 OPENROUTER_API_KEY=[KEY] python3 scripts/eval_grok43_mini.py`
- Use case: Fast feedback on prompt/code changes
- **WARNING:** Mini-eval has ±3–4 NPS variance per persona per run at n=8. Do not optimize on mini-eval alone. Use it to confirm no regression, then run the full eval.

**Full 30-user eval** (`eval_gemma4_qat_v2.py`):
- 30 sessions: 10 grok43 + 10 gpt55 + 10 opus48 simulator groups (opus48 sometimes stops at 5 due to budget)
- User simulators: grok-4.3 (resistant/direct), gpt-5.5 (balanced), claude-opus-4-8 (cooperative)
- QA judge: GPT-5.5
- Cost: ~$3.50–5.00 | Time: ~45–60 min
- Run: `cd /opt/oimy-engine && OIMY_API_URL=http://localhost:8080 OPENROUTER_API_KEY=[KEY] python3 scripts/eval_gemma4_qat_v2.py`
- Use case: Definitive NPS measurement; use for comparing major changes
- Note: Report header says "Gemma 4 12B QAT q4" — hardcoded template. The actual companion model depends on the engine's `COMPANION_MODEL` env var.

### QA Judge Dimensions

| Dimension | Scale |
|-----------|-------|
| Relevance | 1–5 |
| Actionability | 1–5 |
| Constraint Adherence | 1–5 |
| Groundedness | 1–5 |
| Tone/Empathy | 1–5 |
| Rapport Depth | 1–5 |
| Tonal Range | 1–5 |
| Trust Signals | 1–5 |
| Honest/Self-Aware | 1–5 |
| Safety Fails | count |
| Desirability | 1–5 |
| Humor Naturalness | 1–5 (where applicable) |

### Governance Flags

- `tone_mismatch` — model doesn't shift tone when user changes register
- `unsolicited_advice` — offering advice when not asked
- `hallucination` — fake facts, wrong ages, invented capabilities ("I'll remind you")
- `emotional_bypass` — pivoting to advice while user is still in emotional state
- `clinical_language_leak` — "dysregulate", "scaffolding", etc.
- `overpromised_capability` — "I'll follow up", "I'll check in"

### Eval Personas

**8 Mini-eval personas:**

*Resistant (5):*
- Marcus Webb — Black American, reserved teen-dad
- Kevin O'Brien — Irish-American, dry/direct
- Aisha Johnson — Caribbean-American, no fluff
- David Al-Mansouri — Lebanese-American, professional
- James Bergstrom — Midwestern, distrusts soft approaches

*Cooperative (3):*
- Priya Krishnan — Tamil, solo morning parent
- Marisol Ortega — Mexican-American, responds to small wins
- Nalini Mehta — Gujarati, wants validation

**30-user eval personas:**
Personas 0–29 loaded from persona file in the eval script. Groups by simulator:
- 0–9: grok43 (resistant/direct)
- 10–19: gpt55 (balanced)
- 20–29: opus48 (cooperative)

**Key observation on grok43 resistant users:**
Marcus Webb, Kevin O'Brien, Aisha Johnson, David Al-Mansouri, James Bergstrom always appear in the grok43 group. The grok43 simulator plays them as resistant, terse, and skeptical. This is the hardest group and the primary signal for whether the system is improving.

### Workflow Governance Rule

A change is only considered "good" if:
- Resistant group improves OR holds steady
- Cooperative group holds or improves
- Overall NPS is >= previous overall NPS

Never report a win based on one group improving at the expense of another.

---

## 4. Complete Eval History (Objective)

### 30-User Full Eval History

| Date | Config | Overall NPS | grok43 | gpt55 | opus48 | Notes |
|------|--------|:-----------:|:------:|:-----:|:------:|-------|
| 2026-06-24 | Grok 4.3 (trust bug: always 0) | 3.8 | — | — | — | trust_level=0 routing bug |
| 2026-06-25 | Grok 4.3 (trust fixed) | 6.0 | — | — | — | Pre-V2 prompt |
| 2026-06-26 | Grok 4.3 + loops v2 (h30 n=30) | 5.97 | 4.5 | — | — | — |
| 2026-06-26 | Grok 4.3 no loops | 5.93 | — | — | — | Loops neutral |
| 2026-06-26 | Grok 4.3 + loops + style | **6.43** | 5.1 | — | — | Best pre-routing-fix |
| 2026-06-26 | Grok 4.3 + KB | 6.03 | — | — | — | KB hurt −0.40, disabled |
| 2026-06-26 | eval_a: routing fixes only | 5.3 | ~3.5 | — | — | Routing fixed but dropped |
| 2026-06-26 | eval_b: + loop threshold 5 | 5.17 | — | — | — | Too strict, −0.13 |
| 2026-06-26 | eval_c: + memory tier 2 | 5.17 | — | — | — | Neutral cold-start |
| 2026-06-28 | Gemma 4 QAT q4 (n=25) | 4.56 | 3.6 | 5.7 | 4.2 | Not competitive |
| 2026-06-29 | Grok 4.3 + session suggestions | **6.0** | 5.6 | 6.7 | 5.4 | +0.7 over eval_a |
| 2026-06-29 | + ctx window + eval_d | 5.87 | 5.0 | 6.3 | 6.3 | Name errors appeared |
| 2026-06-29 (final) | All 4 fixes | TBD (eval running) | — | — | — | Expected 6.5–7.0 |

**Note on baselines:** The June 6 clean baselines (Grok 4.3: 7.4, Gemma 4 QAT: 7.3) used different eval conditions. eval_a (5.3) is the apples-to-apples baseline using the current eval harness and should be used for all comparisons going forward.

### Mini-Eval History

| Run | Config | Resistant avg | Cooperative avg | Overall |
|-----|--------|:---:|:---:|:---:|
| Mini-eval 1 | Original + anti-rep + terse + deliverable | 5.2 | 5.67 | 5.38 |
| Recheck | + explicit-request exception | 4.5 | 5.0 | 4.75 |
| Mini-eval 2 | + practical-mode + explicit-pain rules | 3.6 | 5.0 | 4.12 |
| Mini-eval 3 | + emotional exemption to anti-rep | 3.8 | 4.33 | 4.00 |
| Mini-eval 4 | Reverted to mini-eval 1 + topic fix | 3.0 | 5.67 | 4.00 |
| Mini-eval 5 (ctx window fix) | All prior fixes | 4.40 | 7.67 | 5.62 |
| Mini-eval 6 (eval_d initial) | + PROBE_FIRST | 4.40 | 8.00 | 5.75 |
| Mini-eval 7 (name fix, PROBE_FIRST loose) | All fixes | 3.40 | 6.33 | 4.50 |
| Mini-eval 8 (PROBE_FIRST tightened) | All 4 fixes | **5.40** | **7.33** | **6.12** |

---

## 5. What Worked (Objective)

### Session Suggestion Tracking (+0.7 NPS in 30-user eval)

**What:** After each companion response, extract the closing action/suggestion, store per-session in `h30_profile.session_suggestions`. Inject as `[Already suggested this session: X, Y, Z]` in context.

**Why it worked:** The model was re-offering the same advice because nothing told it what it had already suggested. The cap (8 suggestions, dedup by 40-char prefix) prevents the most common repetition failure.

**Evidence:** grok43 resistant group: 3.0 → 5.6 NPS in 30-user eval. Nalini Mehta individual NPS: 2–4 → 8.

### [CURRENT TOPIC] Reordering

**What:** Topic summary injected AFTER [FAMILY] entity context, not before.

**Why it worked:** Models give recency weight to the last-seen context. [FAMILY] appearing last caused the model to re-anchor to early-session topics even when [CURRENT TOPIC] said something different.

### PROBE_FIRST Mode (tightened version)

**What:** When advice hasn't landed — either via explicit constraint rejection + suggestions made (Signal A), or 3 consecutive ≤10-word messages with no ack markers + no emotional content + suggestions made (Signal B) — skip advice for one turn. Show you understand, ask ONE targeted question.

**Why it worked:** Kevin O'Brien NPS: 3 → 7. David Al-Mansouri: 3–4 → 7. These users were receiving advice before feeling understood. The probe turn unlocks honest context from the user, enabling more targeted follow-up.

**The tightening that fixed regressions:** The initial version used Signal B with 2 messages and a 12-word limit, which misfired on Marisol Ortega (NPS 4 vs 8 expected). Tightening to 3 messages, 10-word limit, plus an emotional topic exemption fixed the regression.

### Context Window Fix

**What:** DB history fetch `LIMIT 16` → 24. Both companion paths: keep last 8 turns verbatim, condense earlier turns to 200 chars (raised from 120 chars after initial fix to preserve name/intro context).

**Why it worked:** At turn 12+, early turns were being dropped entirely. Parents introduce themselves and their family in turns 1–2; losing this caused context drift and name confusion.

### [FAMILY] Disambiguation Fix

**What:** Changed label from `[FAMILY]` to `[FAMILY — people in this parent's household. These are family members being discussed, NOT the parent you are speaking with.]`. Added user's own name as first line of `[PROFILE]` context: `"You are speaking with: [name]"`.

**Why it worked:** Model was calling Priya Krishnan and David Al-Mansouri by their child's name at turn 12. Entity context listed child names but had no disambiguation vs the parent. With condensed early turns (where "I'm Priya, my son is Rohan" appeared), the model confused which name was the user.

---

## 6. What Did Not Work (Objective)

### Prompt Additions Beyond Mini-Eval 1 State

Every prompt addition after mini-eval 1 — practical-mode rules, explicit-pain-request rules, emotional exemption to anti-repetition rule — degraded performance or was neutral. The emotional exemption was actively harmful: resistant users often have emotional topics even with practical communication styles, so the exemption effectively disabled the anti-repetition rule for exactly the users who needed it most. **The original prompt (mini-eval 1 state) is the best known prompt state.**

### Memory Tier 2 (Confidence Decay)

Eval_c showed no improvement over eval_a on cold-start eval sessions. Expected — cross-session memory requires prior history. For real users with multiple sessions, this may perform better, but it cannot be measured with the current eval harness.

### KB (Knowledge Base) Injection

Hurt NPS by −0.40. Already disabled in production via `if False:` guard. Do not re-enable.

### Gemma 4 QAT q4 (4-bit quantization)

4.56 NPS vs Grok 4.3's 6+ with the same improvements. Not competitive for production.
Note: Gemma 4 Q4_K_M quantization (a different quant) scored 6.82 in a June 8 eval — worth revisiting if pursuing a local model strategy.

### Loop Threshold = 5

Eval_b showed −0.13 vs eval_a. Too strict — prevented loop context injection that was helpful. Default threshold = 2 is better.

---

## 7. Current Failure Patterns (Data-Driven)

### Governance Flag Counts (Last 30-User Eval, 360 Turns)

| Flag | Count | Rate |
|------|-------|------|
| tone_mismatch | 89 | 24.7% |
| unsolicited_advice | 70 | 19.4% |
| hallucination | 48 | 13.3% |
| emotional_bypass | 27 | 7.5% |
| clinical_language_leak | 23 | — |
| overpromised_capability | 14 | — |

### Persistent Low-NPS Personas

**Marcus Webb (NPS 3–4):** Misses multi-topic shifts within a session. When conversation moves from topic A → B → C, the model sometimes stays on A or returns to A. The `[CURRENT TOPIC]` summary should catch this but may not update fast enough (updates every 3 turns). Marcus's typical pattern: starts with meal planning → shifts to talking about son Tyrone's withdrawal → shifts to own feelings of inadequacy → OiMy returns to meal plans.

**Aisha Johnson (NPS 2–3):** Doesn't complete explicit task deliverables. When she asks "write me an email" she sometimes gets advice about what to write instead of the email. `TASK_EXECUTE` mode should catch this but the pattern detection may miss her specific phrasing patterns.

### QA Dimension Weaknesses (Last 30-User Eval)

| Dimension | Score | Target |
|-----------|-------|--------|
| Rapport Depth | 3.66/5 | 4.0+ |
| Tonal Range | 3.56/5 | 4.0+ |
| Actionability | 3.78/5 | 4.0+ |
| Constraint Adherence | 3.74/5 | 4.0+ |

---

## 8. Hypotheses Not Yet Tested

### For Marcus Webb (Multi-Topic Tracking)

Marcus's failure pattern: starts with meal planning → shifts to talking about his son Tyrone's withdrawal → shifts to his own feelings of inadequacy → OiMy returns to meal plans. The `[CURRENT TOPIC]` summary should catch this, but may not update fast enough (updates every 3 turns).

**Hypothesis:** A per-turn topic tracking approach — appending the user's last message topic directly into the context rather than relying on a 3-turn summary — could help. Alternatively, explicitly marking topic shifts in the context block when detected.

### For Aisha Johnson (Task Completion)

Aisha explicitly asks for deliverables but sometimes gets advice. The `TASK_EXECUTE` mode should catch this but the pattern detection may miss her specific phrasing. Worth reviewing:
- What exact phrases she uses in the eval sessions
- What the `TASK_EXECUTE` regex currently matches
- Whether the gap is in detection or in the model ignoring the mode instruction

### For Tonal Range Improvement

The model reads as "warm advisor" in almost all contexts. Real range would mean: punchy 7am text ("Try a choice-of-two: X or Y?") vs measured evening ("That sounds really hard. Let's think about this together.") vs playful when the user is joking. This likely requires model-level capability that prompt changes alone may not deliver.

### For Hallucination Reduction

The model invents ages, names wrong family members, and promises capabilities (reminders) it doesn't have. The `[Facts only from this conversation]` rule is in the prompt but not consistently followed. Adding an explicit, near-the-top rule — "do not invent ages or dates. Do not promise to follow up, remind, or check in. Only reference facts the parent explicitly told you." — may help without harming other dimensions.

---

## 9. Illustrative End-to-End Example

**Scenario:** Kevin O'Brien (resistant, direct) at turn 6, after OiMy has suggested "build-a-bowl dinners" twice and Kevin gave short, non-engaged responses.

**Turn 6 user input:** `yeah that won't work`

**Context assembly (what the model actually sees):**

```
[SYSTEM]
You are Oi — a warm, sharp, emotionally intelligent family companion...
[201 lines of companion prompt]
[COMPANION_ADDENDUMS: anti-repetition, terse-trend, complete-deliverable rules]

[TURN MODE: PROBE_FIRST] The approach so far hasn't resonated — the user is giving
brief or flat responses after advice. Skip any suggestion this turn. Lead with exactly
ONE sentence showing you understand their specific situation. Then ask ONE targeted
question: what have they already tried, or what outcome matters most to them right now.
No advice, no tips, no reframes this turn. One question only.

---

[FAMILY — people in this parent's household. These are family members being discussed,
NOT the parent you are speaking with.]
Child: Emma (age 8) — won't eat vegetables or mixed textures
Child: Sophie (age 8) — won't eat vegetables or mixed textures

[PROFILE]
[User] You are speaking with: Kevin
[Facts]
Barrier (dinners): twin girls with opposite food preferences, won't eat same thing

[Already suggested this session]
* Build-a-bowl approach: separate components, let each choose
* Taco variation for Emma and Sophie

[CURRENT TOPIC — what this conversation is specifically about right now: Kevin is trying
to find dinner solutions that work for both Emma and Sophie given their strong and
opposite food preferences]

[Conversation history — turns 1-4 condensed to 200 chars]
user: My twin girls won't eat the same thing for dinner. Emma likes bland food,
      Sophie likes everything spiced. Makes cooking a nightmare every night. Both are 8…
assistant: That's a real daily drain. Build-a-bowl is worth trying — base is the same…
user: we tried that, they argue about who gets the good stuff
assistant: Got it. Could do a taco bar variation — Emma gets plain…
user: yeah we do tacos sometimes

[Conversation history — last 8 turns verbatim]
user: doesn't really scale to weeknight cooking though
assistant: Fair. What's the constraint — time, groceries, both? [...]
user: yeah that won't work
```

**Expected PROBE_FIRST response:**

> "Two picky 8-year-olds with opposite tastes on a weeknight is a genuinely hard puzzle — sounds like you've already tried the separation approaches. What's the one constraint you most want to avoid: the cooking complexity, the separate-plate logistics, or the arguing?"

**Why this works:**
- Shows understanding of their exact situation (not generic empathy)
- Doesn't repeat build-a-bowl or taco (both in [Already suggested])
- Asks ONE question that will unlock the next, more targeted suggestion
- Doesn't feel like therapy or parenting advice

**What would have happened without PROBE_FIRST:**
Normal default mode would generate another dinner suggestion, possibly a reworded version of build-a-bowl. Kevin would give another short non-engaged response. NPS: 3.

---

## 10. Architecture Diagram (Text)

```
                    User (Telegram / iOS)
                           │
                    POST /chat
                           │
                   OiMyHandler.do_POST()
                           │
                   ┌───────▼────────┐
                   │  OiMyEngine    │
                   │  process_chat  │
                   └───────┬────────┘
                           │
              ┌────────────┼────────────┐
              │            │            │
         ┌────▼───┐  ┌─────▼──────┐ ┌──▼──────────┐
         │H30Engine│  │Entity      │ │HonchoMemory │
         │.tick()  │  │Extractor   │ │.get_insight()│
         │         │  │.get_ctx()  │ │             │
         └────┬────┘  └─────┬──────┘ └──┬──────────┘
              │             │            │
              └────────────▼────────────┘
                           │
                  ┌────────▼────────┐
                  │_classify_turn_  │
                  │mode()           │
                  │BE_PRESENT?      │
                  │PROBE_FIRST?     │
                  │TASK_EXECUTE?    │
                  │JUST_RESPOND?    │
                  └────────┬────────┘
                           │
                  ┌────────▼────────┐
                  │_get_rolling_    │
                  │topic_summary()  │
                  │(GPT-5.4-mini,   │
                  │ every 3 turns)  │
                  └────────┬────────┘
                           │
                  ┌────────▼────────┐
                  │Companion call   │
                  │Grok 4.3 via     │
                  │OpenRouter       │
                  │                 │
                  │System prompt:   │
                  │ companion_v2.txt│
                  │ + ADDENDUMS     │
                  │ + [FAMILY]      │
                  │ + [PROFILE]     │
                  │ + [HONCHO]      │
                  │ + [suggested]   │
                  │ + [TOPIC] ←last │
                  │                 │
                  │History:         │
                  │ last 8 verbatim │
                  │ older → 200char │
                  └────────┬────────┘
                           │
                  ┌────────▼────────┐
                  │Post-processing  │
                  │H30.tick()       │
                  │extract_sugg()   │
                  │add_session_     │
                  │suggestion()     │
                  │Honcho.record()  │
                  └────────┬────────┘
                           │
                    Response to user
```

---

## 11. Credentials & Access

> **SECURITY WARNING:** This section contains live production credentials. Do not post publicly, commit to a public repo, or share the URL broadly. Treat this document like a password manager entry.

### Server Access

| Field | Value |
|-------|-------|
| Host | `root@116.203.107.13` |
| Auth | Password-based SSH |
| SSH password | Provided by Bharath separately — not stored here |
| Alternative | Add your SSH public key to `/root/.ssh/authorized_keys` |

### API Keys

| Service | Key |
|---------|-----|
| OpenRouter (primary) | `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` |
| Honcho | `hch-v3-q2a6kxbivf1f60gqru2zelivokr55pdihe6cavo8zi1a9pqik661uslipkei8rir` |
| OpenAI direct | `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` |
| ElevenLabs | `sk_5984e63cdbd12ae10cb9b55af7c11830a40ef782ce5f2466` |
| Composio | `ak_bg6f-Ju-uJVXfzq8yxFj` |

### GitHub

| Field | Value |
|-------|-------|
| Repo | `https://github.com/aahalife/oimy-skill-engine` |
| PAT | `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` |
| Clone command | `git clone https://oauth2:[PAT]@github.com/aahalife/oimy-skill-engine.git` |

### Local Development Paths (on Hetzner VM)

| Path | Purpose |
|------|---------|
| `/opt/oimy-engine/` | Live engine (edit here for immediate effect) |
| `/home/exedev/oimy/oimy-skill-engine/` | Repo clone (for docs/git) |
| `/opt/oimy-engine/data/oimy.db` | SQLite database |
| `/opt/oimy-engine/docs/synthetic-user-testing/reports/` | All eval reports |

---

## 12. Running Evals

### Quick Workflow

```bash
# SSH to server
ssh root@116.203.107.13

# Verify engine is running
systemctl status oimy-engine

# Run mini-eval (fast check, ~15 min, ~$1)
cd /opt/oimy-engine
OIMY_API_URL=http://localhost:8080 \
OPENROUTER_API_KEY=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 \
python3 scripts/eval_grok43_mini.py

# Run full 30-user eval (~60 min, ~$4)
OIMY_API_URL=http://localhost:8080 \
OPENROUTER_API_KEY=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 \
python3 scripts/eval_gemma4_qat_v2.py

# Restart engine after code changes
systemctl restart oimy-engine
systemctl status oimy-engine
journalctl -u oimy-engine -n 50 --no-pager
```

### Change → Test Cycle

```bash
# 1. Edit files on server at /opt/oimy-engine/
#    Always backup first:
cp /opt/oimy-engine/api.py /opt/oimy-engine/api.py.bak.DESCRIPTION

# 2. Make your changes, then restart
systemctl restart oimy-engine

# 3. Check startup (look for errors)
journalctl -u oimy-engine -n 20 --no-pager

# 4. Run mini-eval
# 5. If mini-eval is good → run full 30-user eval
# 6. If results are good → commit and push

# Commit workflow
cd /home/exedev/oimy/oimy-skill-engine/
git add -p
git commit -m "Description of change + NPS result"
git push
```

### Decision Rules

| Condition | Action |
|-----------|--------|
| Mini-eval resistant drops >0.5 | Stop, investigate before full eval |
| Mini-eval cooperative drops >1.0 | Stop, investigate |
| Mini-eval overall improves >0.3 | Run full 30-user eval |
| Full eval resistant improves OR holds, overall ≥ prev | Commit and document |
| Full eval overall drops | Revert, analyze failure pattern |

---

## 13. Key Architectural Assumptions

1. **Eval measures cold-start.** Eval sessions start fresh with no prior user history. Real users have Honcho memory + entity facts from prior sessions. Real-world NPS is likely higher than eval NPS. Do not conflate them.

2. **grok43 simulator is the hardest.** The Grok 4.3 simulator plays resistant/terse users more authentically than GPT-5.5 or Opus 4.8. grok43 group NPS is the most diagnostic signal. If grok43 improves, it's a real improvement.

3. **Session suggestions cap at 8.** The deduplication uses a 40-char prefix match on the last suggestion. This may miss semantic duplicates with different phrasing. Do not exceed 8 or the dedup window becomes meaningless.

4. **PROBE_FIRST is one-turn-only.** It doesn't persist across turns. If the user's next response re-engages (longer, ack marker), normal mode resumes automatically. This is intentional — sustained probe mode would feel interrogative.

5. **Honcho is only useful after multiple real sessions.** Cold-start eval sessions get empty Honcho responses. Do not evaluate Honcho effectiveness via the eval harness; it will always appear neutral.

6. **The eval report header says "Gemma 4 12B QAT q4."** Hardcoded template. Ignore it. The actual companion model is whatever `COMPANION_MODEL` is set to in the engine env. Check `systemctl cat oimy-engine` or `/opt/oimy-engine/.env` to confirm the live model.

7. **Cooperative group is your canary.** When making changes targeting resistant users, always verify the cooperative group (Priya, Marisol, Nalini) didn't regress. Cooperative users are sensitive to over-engineering — if the system becomes too formulaic, their NPS drops even when resistant users improve.

8. **Prompt additions tend to hurt, not help.** The empirical record across mini-evals 2–4 shows that adding rules to the companion prompt consistently degraded performance. Changes to code logic (turn mode detection, context ordering, suggestion tracking) have shown more durable gains than prompt additions.

---

*Document prepared: 2026-06-29. Last eval run: TBD (eval_d final, all 4 fixes). Point of contact: Bharath (bharath.mech@gmail.com).*