Architecture Deep Dive

How OiMy thinks,
remembers, and responds

Every message you send travels through 8 intelligent layers — all in under 2 seconds.

Live on Hetzner · gpt-5.4-mini router
89 specialist skills
Grok 4.3 companion

The 8-Layer Journey

From your words to a warm, contextual response — here's what happens inside.

📱
Layer 1
Your Message Arrives
You send a message from Telegram, WhatsApp, or the iOS app. It arrives at OiMy's API server (Hetzner, Germany) as a plain text string with your user ID.
What it carries: your message text, your user ID (to load your family profile), the channel source (Telegram/WhatsApp/iOS), and a timestamp. Nothing else — no interpretation yet.
Telegram Bot API WhatsApp Cloud API iOS Webhook
🎯
Layer 2 · ~80ms
Intent Router — Picks the Right Skill
This is the brain's first decision: what kind of help does this person need? OiMy has 89 specialist skills — each one handles a specific domain (parenting, medications, groceries, fitness, emotional support, and more). The router reads your message and picks exactly one.
How it works (two passes):

Pass 1 — Rule engine (instant): Keyword patterns decide obvious cases. "remind me to" → oimy-reminders. "add eggs to" → shopping-list. If an active conversation thread exists (you were just talking about school), it also considers continuing that thread.

Pass 2 — LLM classifier (if needed): Ambiguous messages go to a small, fast model. It reads a 296-line disambiguation guide plus a directory of all 89 skills and their trigger descriptions. Picks one skill ID. Returns it as a single token in ~80ms.

Disambiguation rules example: "set up a daily check-in for my dad" → the rules say if it mentions scheduling + elder, pick elder-checkin, not elder-care. These rules were hand-tuned for the exact failure cases found in testing.
☁️ gpt-5.4-mini (cloud) 🏠 Gemma 4 12B QAT (local)
🧠
Layer 3 · ~60ms
Context Assembly — Gathers What Matters
OiMy knows your family. Before any LLM sees your message, the system assembles everything relevant into a structured context block — your family members, their ages and needs, what you've said before, how you communicate, and what situations are currently live.
Three sources of context (injected every turn):

① Entity Facts (Tier 1 — always injected): Names, ages, relationships, medical conditions, allergies, school schedules. These are permanent facts extracted from past conversations. Example: "Mia (daughter, 2y), Pablo (son, 5y), husband travels for work."

② Psych Profile: Communication style (direct vs. venting-first), engagement level, parenting philosophy, emotional state patterns. Determined from conversation history using a personality inference model.

③ Active Loops (up to 2 injected): OiMy tracks 84 ongoing "life situations" — things like sleep regression, school transition, aging parent, fitness goal, or relationship stress. If a loop is active, a one-line context note is added ("Mia is going through sleep regression — do not offer new sleep strategies if already discussed").

④ Honcho Memory: Relevant past messages retrieved from a cloud conversation store using semantic similarity — so OiMy remembers what you said 3 weeks ago if it's relevant now.
🗄 SQLite entity_facts 💾 Honcho cloud memory 🔄 Loop state DB
⚙️
Layer 4 · ~20ms
Skill Execution — The Right Specialist Takes Over
Now the identified skill builds its own prompt — a focused, role-specific instruction set that tells the LLM exactly how to respond for this type of request. Think of each skill as a specialist who knows their domain deeply.
Four workflow types:

🗣 Coach — Conversation-first. Asks clarifying questions, validates feelings, builds toward a plan. Used for parenting advice, life coaching, emotional support, relationship guidance. (~40 skills)

⚡ Action — Execute immediately. Parses structured data (times, items, names), calls APIs, confirms result. Used for reminders, grocery lists, appointments, medication tracking. (~30 skills)

🏥 Expert — Evidence-based, careful, non-prescriptive. Provides information with clear boundaries ("see a doctor for diagnosis"). Used for medical symptoms, legal questions, financial decisions. (~10 skills)

💬 Chat — General conversation fallback. Warm, human, no specific structure. Used when nothing else matches clearly.
89 skill manifests 4 workflow types BCT engine
💡
Layer 5 · ~800–1400ms
LLM Inference — The Heart of the Response
The assembled prompt — your family context + skill instructions + your current message — goes to the companion model. This is where the actual thinking happens: understanding nuance, matching tone, generating a response that feels human and warm, not robotic.
What the model receives (assembled prompt):
System role + skill instructions → family context block (Tier 1 facts) → psych profile → active loop notes → your message history (last few turns) → your current message.

What it must do: Follow the skill's workflow type, honor communication style preferences (no unsolicited advice if the user is venting), stay within the skill's topic boundaries, and sound like a wise friend — not a chatbot.

Fallback chain: If Grok 4.3 is unavailable or times out (8s budget), Claude Haiku 4.5 takes over automatically.
☁️ Grok 4.3 (primary) 🏠 Gemma 4 12B QAT (local) 🔄 Claude Haiku 4.5 (fallback)
Layer 6 · ~10ms
Quality & Safety Gate
The generated response passes through a fast rule-based quality layer before being sent. This catches common failure modes without needing another LLM call.
What gets checked:

Unsolicited advice: Did the model give advice when the user was just venting? Flag and soften.
Overpromising: Did the model claim to do something it can't ("I'll book that now")? Remove capability inflation.
Safety signals: Any indicators of self-harm, domestic violence, or child crisis? Escalate to emergency protocols.
Null action failures: Did a reminder get saved without a time? Ask the user instead of saving "None".
Tone mismatch: Did the response feel robotic or list-heavy when the user needed warmth?
quality_rules.py safe_response.py safety_gates.py
💾
Layer 7 · async
Memory Update — OiMy Gets Smarter
While you're reading the response, OiMy is updating its memory. Facts, patterns, and loop states are written back — so next time, it knows more about your family.
Three memory stores updated:

① Entity Facts (SQLite): New names, ages, health conditions, or preferences mentioned this turn get extracted and stored. Confidence scores track reliability. Old facts get a time-based decay (more recent = higher confidence).

② Loop State (SQLite): If a life-situation loop was active, the system records what was addressed. Loops can go dormant (after their natural phase ends) and reactivate if the situation returns.

③ Honcho Conversation Store: The turn is saved to Honcho's cloud memory, making it retrievable in future sessions via semantic search. OiMy never forgets — it just prioritizes what's most relevant.
🗄 oimy.db (SQLite) ☁️ Honcho cloud memory
💬
Layer 8 · <2 seconds total
Your Response — Warm, Contextual, Human
The response is sent back to your device. For Telegram, that's a message (and sometimes inline quick-reply buttons for common next steps). For WhatsApp, a formatted message. For iOS, a push notification that opens the conversation.
What makes it feel different from a chatbot: OiMy called you by name, referenced your daughter Mia specifically, didn't give you a 5-point listicle when you needed empathy, and remembered that you've been stressed about sleep regression for 3 weeks. None of that required you to repeat yourself.
Telegram Bot API WhatsApp Cloud API iOS APNs

User Message

"My toddler won't stop crying and I don't know what to do 😭"
1
Input Layer
Message arrives from Telegram user ID 8040940969. Timestamp: 14:33 UTC. Channel: Telegram.
2
Intent Router
Rule engine: no keyword match. LLM classifier reads disambiguation rules — sees "won't stop crying" + "I don't know what to do" → crisis framing, not strategy question. Picks patpat (immediate parenting crisis), not parenting-coach (which would apply if asking "how do I handle tantrums?"). Latency: 78ms.
3
Context Assembly
Entity facts: Mia (daughter, 2y 4m), Pablo (son, 5y). Active loop: LP-001 Sleep Regression (activated 18 days ago — Mia has had disrupted sleep). Psych profile: high emotional sensitivity, prefers validation before advice. Honcho: last 4 relevant turns loaded (3 prior patpat sessions with Mia this month).
4
Skill: patpat
Workflow type: coach, crisis mode. Skill instruction: validate first, do NOT jump to solutions, ask one focused question if needed. BCT engine: distress + helplessness detected → apply "grounding + normalizing" technique (BCT 15.1). Do NOT suggest new sleep strategies (loop says they've already been discussed).
5
LLM: Grok 4.3
Generates: "That sounds so hard — especially with Mia being in her sleep regression phase. You're doing fine, even when it doesn't feel that way. Is she upset about something specific, or is this just one of those no-reason meltdowns?" — 42 tokens, 910ms.
6
Quality Gate
No unsolicited advice ✓. No overpromising ✓. No safety signals ✓. Tone: warm and open-ended ✓. Passes.
7
Memory Update
Loop LP-001 event recorded: crisis_support_given. Entity: "Mia distress episode" added with confidence 0.7. Honcho: turn saved. No new entity facts extracted (no new names/ages mentioned).
8
Response Delivered
Telegram message sent in 1.2 seconds total. Warm, specific to Mia, no generic parenting advice, invites the parent to share more.

Companion · Cloud

Grok 4.3
Primary response model. Strong at nuanced, emotionally intelligent coaching. Via OpenRouter.

Companion · Local

Gemma 4 12B QAT
Local deployment model. Runs on Apple Silicon or Windows GPU. Requires think:false for routing. Same NPS as cloud.

Intent Router · Cloud

gpt-5.4-mini
Fast, cheap classifier. Reads 89-skill directory + 296-line disambiguation rules. 65% routing accuracy. ~80ms.

Intent Router · Local

Gemma 4 12B QAT
Dual-use: same model as companion. 69% routing accuracy (beats gpt-5.4-mini). Requires think:false flag.

Fallback Companion

Claude Haiku 4.5
Auto-fallback if Grok 4.3 times out (>8s) or is unavailable. Warm, reliable, lower latency.

Memory Inference

Honcho
Dialectic retrieval for conversation history. Returns semantically relevant past turns without injecting everything.