1 What Is OiMy

OiMy is a parenting companion app. Users text via Telegram (or an iOS app). OiMy's AI companion — called "Oi" — responds like a trusted friend: warm, practical, not clinical. Users are parents dealing with real family challenges: toddler tantrums, teen withdrawal, co-parenting tension, meal-time battles, homework struggles, and more.

Oi's Voice Contract

Warm but not saccharine. Practical, not preachy. Never clinical. Never patronizing. It should feel like a friend who happens to know a lot about child development — not a therapist or a parenting hotline.

The Engagement Problem

Many parents — especially fathers or reserved parents — resist "soft" coaching approaches. They want solutions, not processing. The system must flex between emotional support mode and practical solution mode based on what the parent actually needs at that moment, not what the system defaults to.

2 System Architecture

Infrastructure

Key Files

Model Stack (Current Production)

Request Flow (chat_light path)

User message arrives via POST /chat
→ OiMyHandler.do_POST()
→ OiMyEngine.process_chat()
  → H30Engine.tick()                      -- update engagement signals
  → EntityExtractor.get_context_summary() -- pull entity context
  → _build_profile_context()              -- name, facts, psych, inferences, H30
  → HonchoMemory.get_insight()            -- cross-session insight
  → Fetch conversation history            -- last 24 rows from recent_messages
  → _extract_session_facts()
  → _classify_turn_mode()                 -- BE_PRESENT / PROBE_FIRST / PLAN_FORGE /
                                          -- TASK_EXECUTE / JUST_RESPOND / VENTING / default
  → _get_rolling_topic_summary()          -- GPT-5.4-mini, every 3 turns
  → _call_openrouter_companion()          -- Grok 4.3 via OpenRouter
      system = companion_v2.txt + ADDENDUMS + system_additions
      context = [DoNotSuggest] + [FAMILY] + [PROFILE] + [HONCHO]
              + [Already suggested] + [CURRENT TOPIC]  ← LAST
      history = last 8 verbatim + older → 200 chars
  → Post-process response
  → H30Engine.tick()                      -- post-turn update
  → _extract_suggestion_made()
  → H30Engine.add_session_suggestion()
  → HonchoMemory.record_turn()
  → Store in recent_messages
→ Return response to user

Turn Mode Detection (_classify_turn_mode)

Priority order — first match wins:

Context Injection Order (Critical for Recency Weight)

LLMs give recency weight to later tokens. This ordering is intentional and must be preserved:

[system prompt]
+ [COMPANION_ADDENDUMS]
+ [other system_additions]
---
[Do Not Suggest]                    ← explicit constraints from user
[FAMILY — family members, NOT the parent]  ← entity context
[PROFILE]                           ← name, facts, psych, inferences, H30
[HONCHO]                            ← cross-session insight
[Already suggested this session]    ← suggestion dedup
[CURRENT TOPIC]                     ← LAST — highest recency weight
[conversation history]              ← last 8 verbatim + older → 200 chars
[current user message]

3 NPS Evaluation Methodology

Two Harnesses

QA Judge Dimensions

Governance Flags

Eval Personas

Mini-eval — Resistant (5): Marcus Webb (Black American, reserved teen-dad), Kevin O'Brien (Irish-American, dry/direct), Aisha Johnson (Caribbean-American, no fluff), David Al-Mansouri (Lebanese-American, professional), James Bergstrom (Midwestern, distrusts soft approaches)

Mini-eval — Cooperative (3): Priya Krishnan (Tamil, solo morning parent), Marisol Ortega (Mexican-American, responds to small wins), Nalini Mehta (Gujarati, wants validation)

30-user eval: Personas 0–29 from persona file. Groups: 0–9 grok43 (resistant), 10–19 gpt55 (balanced), 20–29 opus48 (cooperative).

4 Complete Eval History (Objective)

30-User Full Eval History

Mini-Eval History

5 What Worked (Objective)

6 What Did Not Work (Objective)

7 Current Failure Patterns (Data-Driven)

Governance Flag Counts (Last 30-User Eval, 360 Turns)

Persistent Low-NPS Personas

Marcus Webb (NPS 3–4): Misses multi-topic shifts within a session. Pattern: starts with meal planning → shifts to son Tyrone's withdrawal → shifts to own feelings of inadequacy → OiMy returns to meal plans. [CURRENT TOPIC] updates every 3 turns and may not track fast enough.

Aisha Johnson (NPS 2–3): Doesn't complete explicit task deliverables. When she asks "write me an email" she gets advice about what to write instead of the email. TASK_EXECUTE mode should catch this but may miss her phrasing patterns.

QA Dimension Weaknesses

8 Hypotheses Not Yet Tested

For Marcus Webb (Multi-Topic Tracking)

The [CURRENT TOPIC] summary updates every 3 turns, which may be too slow for rapid topic shifts. A per-turn topic tracking approach — appending the user's last message topic directly into the context rather than relying on a 3-turn summary — could help, or explicitly marking topic shifts in the context block when detected.

For Aisha Johnson (Task Completion)

The gap may be in detection or in the model ignoring the mode instruction. Worth reviewing: (1) what exact phrases she uses in the eval sessions, (2) what the TASK_EXECUTE regex currently matches, (3) whether adding explicit examples of task phrases ("write me", "draft a", "make me a list") would help.

For Tonal Range Improvement

The model reads as "warm advisor" in almost all contexts. Real range: punchy 7am text ("Try a choice-of-two: X or Y?") vs measured evening ("That sounds really hard. Let's think about this together.") vs playful when joking. This likely requires model-level capability that prompt changes alone may not deliver.

For Hallucination Reduction

Adding an explicit near-the-top rule: "Do not invent ages or dates. Do not promise to follow up, remind, or check in. Only reference facts the parent explicitly told you." Current prompt has a softer version that isn't consistently followed.

9 Illustrative End-to-End Example

[TURN MODE: PROBE_FIRST]
The approach so far hasn't resonated. Skip any suggestion this turn.
Lead with ONE sentence showing you understand their specific situation.
Then ask ONE targeted question: what have they already tried, or what
outcome matters most to them right now. No advice, no tips. One question only.

---

[FAMILY — people in this parent's household, NOT the parent]
Child: Emma (age 8) — bland food, won't eat vegetables or mixed textures
Child: Sophie (age 8) — opposite preferences, likes spiced food

[PROFILE]
[User] You are speaking with: Kevin
[Facts] Barrier (dinners): twin girls with opposite food preferences

[Already suggested this session]
* Build-a-bowl approach: separate components, let each choose
* Taco variation for Emma and Sophie

[CURRENT TOPIC] Kevin is trying to find dinner solutions for both Emma and
Sophie given their strong and opposite food preferences

[History — condensed]
user: My twin girls won't eat the same thing. Emma likes bland, Sophie likes spice...
assistant: That's a real drain. Build-a-bowl is worth trying...
user: we tried that, they argue about who gets the good stuff
assistant: Could do a taco bar variation...
user: yeah we do tacos sometimes
assistant: Fair. What's the constraint — time, groceries, both?
user: yeah that won't work

10 Architecture Diagram

                    User (Telegram / iOS)
                           │
                    POST /chat
                           │
                   OiMyHandler.do_POST()
                           │
                   ┌───────▼────────┐
                   │  OiMyEngine    │
                   │  process_chat  │
                   └───────┬────────┘
                           │
              ┌────────────┼────────────┐
              │            │            │
         ┌────▼───┐  ┌─────▼──────┐ ┌──▼──────────┐
         │H30Engine│  │Entity      │ │HonchoMemory │
         │.tick()  │  │Extractor   │ │.get_insight()│
         │         │  │.get_ctx()  │ │             │
         └────┬────┘  └─────┬──────┘ └──┬──────────┘
              │             │            │
              └────────────▼────────────┘
                           │
                  ┌────────▼────────┐
                  │_classify_turn_  │
                  │mode()           │
                  │BE_PRESENT?      │
                  │PROBE_FIRST?     │
                  │TASK_EXECUTE?    │
                  │JUST_RESPOND?    │
                  └────────┬────────┘
                           │
                  ┌────────▼────────┐
                  │_get_rolling_    │
                  │topic_summary()  │
                  │(GPT-5.4-mini,   │
                  │ every 3 turns)  │
                  └────────┬────────┘
                           │
                  ┌────────▼────────────────────────┐
                  │ Companion call — Grok 4.3        │
                  │ via OpenRouter                   │
                  │                                  │
                  │ System prompt:                   │
                  │   companion_v2.txt               │
                  │   + ADDENDUMS                    │
                  │   + [FAMILY]                     │
                  │   + [PROFILE]                    │
                  │   + [HONCHO]                     │
                  │   + [Already suggested]          │
                  │   + [CURRENT TOPIC]  ← LAST      │
                  │                                  │
                  │ History:                         │
                  │   last 8 verbatim                │
                  │   older → condensed to 200 chars │
                  └────────┬────────────────────────┘
                           │
                  ┌────────▼────────┐
                  │Post-processing  │
                  │H30.tick()       │
                  │extract_sugg()   │
                  │add_session_     │
                  │  suggestion()   │
                  │Honcho.record()  │
                  └────────┬────────┘
                           │
                    Response to user

11 Credentials & Access

Server Access

API Keys

GitHub

Local Development Paths (on Hetzner VM)

12 Running Evals

Quick Workflow

# SSH to server
ssh root@116.203.107.13

# Verify engine is running
systemctl status oimy-engine

# Run mini-eval (~15 min, ~$1)
cd /opt/oimy-engine
OIMY_API_URL=http://localhost:8080 \
OPENROUTER_API_KEY=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 \
python3 scripts/eval_grok43_mini.py

# Run full 30-user eval (~60 min, ~$4)
OIMY_API_URL=http://localhost:8080 \
OPENROUTER_API_KEY=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 \
python3 scripts/eval_gemma4_qat_v2.py

# Restart engine after code changes
systemctl restart oimy-engine
systemctl status oimy-engine
journalctl -u oimy-engine -n 50 --no-pager

Change → Test Cycle

# 1. Always backup before editing
cp /opt/oimy-engine/api.py /opt/oimy-engine/api.py.bak.DESCRIPTION

# 2. Edit, then restart
systemctl restart oimy-engine

# 3. Check startup (look for errors)
journalctl -u oimy-engine -n 20 --no-pager

# 4. Run mini-eval
# 5. If mini-eval is clean → run full 30-user eval
# 6. If results are good → commit and push

cd /home/exedev/oimy/oimy-skill-engine/
git add -p
git commit -m "Description + NPS result"
git push

Decision Rules

13 Key Architectural Assumptions

1. Eval measures cold-start. Eval sessions start fresh with no prior user history. Real users have Honcho memory + entity facts from prior sessions. Real-world NPS is likely higher. Do not conflate eval NPS with production NPS.
2. grok43 simulator is the hardest signal. The Grok 4.3 simulator plays resistant/terse users more authentically than GPT-5.5 or Opus 4.8. grok43 group NPS is the most diagnostic signal. If grok43 improves, it's a real improvement.
3. Session suggestions cap at 8. Deduplication uses a 40-char prefix match. May miss semantic duplicates with different phrasing. Do not exceed 8 or the dedup window becomes meaningless.
4. PROBE_FIRST is one-turn-only. Doesn't persist across turns. If the user's next response re-engages (longer, ack marker), normal mode resumes automatically. Sustained probe mode would feel interrogative.
5. Honcho is only useful after multiple real sessions. Cold-start eval sessions get empty Honcho responses. Do not evaluate Honcho effectiveness via the eval harness.
6. The eval report header says "Gemma 4 12B QAT q4." Hardcoded template. The actual companion model is whatever COMPANION_MODEL is set to in the engine env. Run systemctl cat oimy-engine to confirm.
7. Cooperative group is your canary. When making changes targeting resistant users, always verify cooperative group (Priya, Marisol, Nalini) didn't regress. Over-engineering causes their NPS to drop even when resistant users improve.
8. Prompt additions tend to hurt, not help. The empirical record across mini-evals 2–4 shows prompt additions consistently degraded performance. Changes to code logic (turn mode detection, context ordering, suggestion tracking) show more durable gains.