A complete field guide to the architecture — memory, profiling, the skills engine, Hermes, model routing, safety, and where every piece actually came from. Written for someone who wants to genuinely understand this system, not skim a pitch deck.
How to read this. OiMy contains designed-but-unshipped systems sitting right next to live production code. This guide tags every claim with its real status — live, gated off, planned, legacy, or an active research track — so you always know which world you're reading about. Where something isn't known, this guide says so instead of guessing.
Start with Part 1 for the two-minute picture. Everything after that is depth you can explore in any order — jump straight to memory, or skills, or safety, whatever you're curious about first.
What OiMy is, in a form you can read in two minutes and actually walk away understanding.
OiMy is a memory-enabled family AI companion. Per its own production system prompt, it exists for a family's ordinary stress, chronic overload, and crisis-adjacent moments — expected to be specific and useful, not just emotionally warm. It combines conversation, parenting and coaching support, household and life-management skills (reminders, shopping lists, meal plans), and persistent context that carries across sessions — so it remembers that Emma has soccer on Tuesdays, and that Dad usually does pickup. api.py:748-752
The personality users actually meet is called Oi — a warm, knowledgeable friend. Never "the app," never "OiMy," never "as an AI." Expert knowledge is treated as an internal compass, not surface content: the frameworks live inside a reply, never on top of it.
Today, the replies come from a cloud frontier model. A fine-tuned local model exists as a serious, measured research track research track — but it has never answered a real user's message. Hold onto that distinction; it recurs everywhere in this system.
OiMy has two parallel product surfaces, built on different code paths. If you remember nothing else from Part 1, remember this — most confusion about OiMy comes from conflating them.
Surface 1 — the Telegram companion live. This is oimy-engine — a large Python engine (api.py, 4,500+ lines) running on a Hetzner server, reached through a thin gateway called Light Hermes. Per the project's own July 15 architecture brief, this is "what users actually talk to." It's the system that received a rigorous independent code audit on 2026-08-16, and it's the primary subject of this guide.
Surface 2 — the web app at app.oimyai.com live. This is OiMy-Platform — a Next.js application that provisions one isolated OpenClaw agent container per family (19–22 Docker containers on the same Hetzner box as of July 2026), fronted by a separate proxy VM and a management API. This is where the multi-tenant family features live: community skill submission, heartbeat templates, a five-question onboarding interview, morning-brief crons.
"The engine" (Telegram, Grok 4.3, api.py) and "the platform" (web app, OpenClaw containers, Next.js) answer different questions. This guide always tells you which one it's describing.
The best way to understand OiMy is to ride along with one real message, end to end. Every step below is live, confirmed by the 2026-08-16 audit.
/chat endpoint decides continuation vs. new session, runs a narrow deterministic medical-emergency check (chest pain/breathing, fever + stiff neck), and checks for a pending bridge response in flight. api.py:2165-2212skill_id + confidence. The manifest supplies a workflow_type: chat_light, action, coach, expert, planner, monitor, or profile. api.py:2482-2487, 2741-2767_execute() routes to the matching archetype. A direct companion turn goes to Grok 4.3; a coach turn commonly hits a DeepSeek V4 Flash scaffold, or escalates to a Claude Sonnet 4.6 "tall model" path — the model that answers depends on routing, not one universal companion. api.py:6042-6073, 4753, 4803-4885session_suggestions, the full coaching-anchor file, the last eight raw history turns, H30 probing instructions, and — if a crisis pattern matched — a safety-alert addition. api.py:5461-5599Hermes forwards it as a real request → simple single-person ask, no bridge needed → the engine already knows Emma's age and schedule, and that Dad usually handles pickup → Grok 4.3 drafts a reply using that context → the safety check confirms "Emma" is a real, known family member.
Notice what made that reply good: not the model — the memory. That's the thesis of the whole engine, and why the deep dives below start with memory.
Each section below stands alone — jump straight to what you're curious about. Click any heading to expand.
Memory is OiMy's central bet: a companion that doesn't remember isn't a companion. The design stacks five layers, from concrete facts up to long-arc life tracking.
People, concepts, and preferences stored as relationship triples — "Sarah --has_condition--> PCOS" — in SQLite + JSONL via gbrain_lite.py. A nightly "dream cycle" LLM sweep hunts for new entities in recent conversation. It's what recognized Emma in Part 2.
Three confirmed data-quality bugs: a null child age is stored as the literal text "None" entity_extraction.py:131; there's no deduplication, so duplicates accumulate entity_extraction.py:163,281; and only five categories — children, barriers, health, preferences, pets — actually reach the prompt. Extracted adult, schedule, motivator, and location facts are collected but dead on this path entity_extraction.py:249.
Every message+response pair is embedded (all-MiniLM-L6-v2, 384 dimensions), with cosine-similarity search surfacing the most relevant past exchanges as "Relevant Past Context." FTS5 keyword search is the fallback if embeddings are unavailable.
Confirmed bug: the embedding cache key uses only the first 200 characters of a message, while the provider embeds up to 8,000 — two long messages sharing a 200-character prefix collide onto the same cached embedding. vector_memory.py:92,103
The most ambitious layer, and the one that most needs precision: the code exists, the schema is complete, and it is not reaching real users today.
Three depths of knowledge about a person: Surface (explicit onboarding data, confidence 1.0), Inferred (pain points, goals, communication style, 0–0.95, growing with evidence), and Deep (Hughes social needs, Hughes decision style, Maslow level, rapport level L1–L5, formative years, belief system — 0.1–0.9, always re-verified against current behavior, never assumed permanent).
Why it's gated: OIMY_MEMORY_TIER is unset in production, defaulting to Tier 1 — entity facts with confidence ≥0.70 only, no decay, no historical tagging, no psychological context. Tiers 2/3 exist in code but aren't active. Nuance: Tier 1 isn't a global "memory off" switch — vector retrieval and entity context are assembled independently of the tier gate api.py:2412-2470; the tier only gates the profile-fact depth/confidence branches api.py:4407. OiMy remembers plenty at Tier 1 — what it doesn't do yet is psychologize. (Full deep dive on this layer in Profiling.)
Long-arc goal, milestone, and deviation tracking across sessions — "Emma needs to improve SAT math by 120 points," with milestones and progress percentages, in a SQLite journey table. Designed. Not built.
The orchestrated nightly job that would tie everything together: summarize → classify stage of change → GBrain learn → update deep profile → update journey → dream-cycle pattern detection → generate the next session's agenda. Designed. Not built.
| Store | Holds | Owner |
|---|---|---|
SQLite oimy.db | Entity facts, profile, H30 state, recent messages, inference state, outbox | The engine |
SQLite vector_memory.db | 384-dim embeddings for semantic search | The engine path bug: api.py:1846 |
| Postgres 17 | users, instances, user_sessions, user_chat_messages | OiMy-Platform (control plane) |
The vector store's path is computed via a fragile string-replace on oimy.db's path, which on production resolves to a sibling directory rather than a sibling file — backups scoped to the engine's data directory may silently miss the entire vector store.
An internal codename (its full meaning isn't documented anywhere read for this guide). Tracks recent messages, session constraints (a "don't suggest this again" list), suggestion history, and opening-variety state. Confirmed bugs: legacy session_constraints are untimestamped and never expire h30.py:378-413; the reset endpoint clears session_suggestions but not session_constraints api.py:6765; a first-message duplication bug on a user's very first tick h30.py:151,158.
OiMy's most intellectually distinctive subsystem — and the one where honesty matters most, because nearly all of it is gated behind the Tier-1 default described above. The right way to hold it: the system is built to understand people this deeply; today's live production surfaces only the entity layer, not this psychological layer.
Per-message, real-time, zero-LLM (~0ms). Pure Python: pain points via regex, goals via keyword matching, sentiment via scoring, timing via time-of-day analysis. Cheap signals, harvested constantly.
Session-end LLM classification (teacher-authored prompts; production model for these classifications is unverified):
Two more gating frameworks shape how a live profile would be used: Maslow level (physiological / safety / love-belonging / esteem / self-actualization) gates whether OiMy pushes growth goals or offers practical logistics help; rapport levels L1–L5 gate conversational depth and directness — Oi earns depth, it doesn't presume it.
The 4-field cold start. From just name, age, gender, and location, a teacher-authored prompt generates an initial speculative profile — confidence 0.1, explicitly marked as a guess — so OiMy has something to work with on message one. Real signal overwrites it within the first few messages.
The belief-system table (added post-May-17): limiting beliefs, core values, a self-efficacy map, a causal model, change-readiness belief — first computed at message 50, re-run every 30 messages after. The system waits for real evidence before daring to model what someone believes.
All of this is real, schema-complete code. None of it reaches real conversations today, because production runs at memory Tier 1. When you talk to Oi right now, it knows your facts, not your psychology.
OiMy has roughly 80–90 skills (documented counts range 73–83 across dated snapshots; a recent reference says 89, unverified — treat "about 80–90" as the honest number). The naive architecture would be one handler per skill. The original prototype's design doc rejected that in one line: "81 skills would mean 81 handlers — unmaintainable."
Instead, every skill is an instance of one of six behavioral archetypes — six shapes of work:
Production routing is done by GPT-5.4-mini in the cloud — not the on-device Gemma 4 router the original prototype was built to prove out (see Lineage for that story).
confirmed bug Four active routing-guard regexes contain a literal backspace byte (0x08) where \b word-boundary escapes were intended — a classic Python string-escaping trap. Those specific reroute corrections (e.g. "reminder" phrases misrouted to meal-planning) silently never fire. api.py:2699, 2711, 2729, 2733
dead code A knowledge-base injection feature sits behind a literal if False: — present in the code, not live. api.py:2941
Sourcing caveat, stated up front: "Hermes" by name appears nowhere in the 2026-08-16 code audit, whose citations live entirely in api.py/oimy//archetypes/. Everything below comes from the 2026-07-15 architecture brief and a 2026-07-11 internal design doc — accurate as of those dated documents, unverified against the most recent code audit.
The gateway users' messages actually pass through. Deliberately, almost aggressively thin: one job — decide "does this belong to OiMy" (using GPT-5.4-mini) and forward. No memory. No persona. Currently serves Telegram, with iMessage reportedly running as of mid-August, ahead of "planned" status in older docs.
The designed successor would give the gateway a full adaptive persona, fed OiMy's real coaching and knowledge content through a direct code-level hook (pre_llm_call) rather than a stuffed prompt. Forcing constraint: the vendor gateway framework caps personas at 20,000 characters, and OiMy's real coaching/knowledge file is roughly six times that size.
Why deferred, and this is worth admiring: an internal review of six real recent production bugs found only one was actually an architecture gap. Five were ordinary defects in the existing engine. So the team chose to stabilize the core engine first rather than build the next layer on a wobbling one.
A 2026-07-11 internal doc records an explicit architectural ruling: entity facts, deep-profile inference, coaching-anchor retrieval, and the constraint ledger stay owned by the engine, permanently. They will not migrate into Hermes.
The reasoning is concrete, not ideological. Hermes is built on a vendor open-source framework (hermes-agent) that evolves very fast — roughly 1,720 commits, 998 PRs, and 370+ contributors between just two recent minor versions. The project confirmed the Telegram platform module itself relocated between two versions it touched. What's stable is the documented pre_llm_call hook and plain HTTP — so that's all OiMy builds against.
One exception in the other direction: session/turn-boundary bookkeeping should lean on Hermes — letting Hermes' own session store become authoritative for Hermes-routed traffic, retiring the engine's competing bespoke session logic — because that was the one genuine architecture gap in the internal bug review, and Hermes had already solved it well. Keep what you must own; adopt what they've genuinely done better.
Ask "which model is OiMy?" and the honest answer is "which sentence?" This is the live production environment, read directly from /proc/<MainPID>/environ during the 2026-08-16 audit:
COMPANION_MODEL=x-ai/grok-4.3
FALLBACK_COMPANION_MODEL=anthropic/claude-haiku-4-5
ROUTER_MODEL=openai/gpt-5.4-mini
COACH_MODEL=x-ai/grok-4.3 ← set, but never read by the code
DEEPSEEK_MODEL=deepseek-chat
COACH_MODEL arrow is the diagram's whole point: a live environment variable that names a model nobody actually calls.system_additions and request_ctx — silently losing turn-mode instructions, topic context, and correction instructions. api.py:5424-5438, 5648 That last loss is what breaks the governance regeneration pass from Part 2, step 7.patpat parenting-coach turn was answered by deepseek-v4-flash, not Grok.COACH_MODEL is set to x-ai/grok-4.3 in the live environment — and the code never reads it. Coach configuration actually flows through TALL_MODEL api.py:1822, 1841, which is unset, so a code default applies instead. Anyone reading the environment would confidently conclude "the coach runs on Grok" — and be wrong. Included here deliberately, as a lesson: configuration is a claim; only code that reads the variable makes it true.
Related liability: the coach's DeepSeek fallback contains a hardcoded embedded API-key default coach.py:935,938 — not the active credential path today since DEEPSEEK_API_KEY is set, but a silent liability if that variable ever disappears. Rotation recommended by the audit.
Alongside the cloud stack runs OiMy's most patient experiment: fine-tuning a Gemma 4 12B (QAT) model with LoRA, on real production behavior categories, across three rounds.
The honest scoreboard: the local model went from "worse" to "competitive," hasn't yet beaten the incumbent, and has never served a real user's message — the eval runbook explicitly resets production back to Grok after every run. Next planned step (MLX conversion for faster local serving) is blocked on an unavailable dev-machine SSH tunnel.
Why bother at all: cost ($15–40/day running Claude Sonnet 4.6 across the older 22-container OpenClaw fleet), latency (2–5s cloud round-trips vs. 200–500ms on-device), and privacy (sensitive family conversations staying on-device). Those pressures haven't gone away — the track is dormant-at-a-blocker, not abandoned. It lives in the sibling lora-v2 / lora-v3 / reasoning-dataset / oimy-ml-artifacts projects — see Lineage.
Safety in OiMy is mostly mechanical — deterministic code-level checks rather than "we asked the model to be careful." That design choice is right, and much of it works. This section presents the current state plainly: an actively worked-on system with real strengths and two confirmed live defects. Neither is presented to alarm; both are presented because a collaborator deserves the same view the team has.
_roster_unknown_names() — Oi cannot casually invent a child or reference a person the family roster doesn't know._ledger_exclusion_violations() — replies are checked against the family's exclusion ledger (allergens and the like).One reliability caveat: the embedding-bank import is fail-open — if it errors at startup, those checks silently disable rather than blocking boot, and nothing currently monitors for that condition.
Confirmed defect 1 — a broken safety fallback. _execute() takes five arguments and defines no local request_ctx api.py:6042, but a downstream fallback call references request_ctx=request_ctx api.py:6112,6120. That's a NameError, silently swallowed by a broad exception handler, substituting generic static text in place of the intended crisis-specific response. Reproduced in isolated execution: an action workflow correctly vetoed for a crisis/escalation signal fell through to "I'm here for you! What would you like to talk about?" Introduced 2026-07-11 (commit 080094a8); still live as of the August 16 audit.
Confirmed defect 2 — third-person self-harm can miss the synchronous crisis path. Phrases like "kill himself" about a child are correctly classified emergency-family/safety-class-4 chat.py:48,61,164, but that skill is typed workflow_type: coach, and the general chat-light companion fast path runs before coach/bridge dispatch and returns immediately on success api.py:2825,2950. The safety-alert injection only checks _detect_parent_crisis (predominantly first-person). Reproduced with the literal input "My son said he wants to kill himself tonight" — routed correctly, but a failed companion path produced a bridge/hold message with no crisis instruction. First-person parent self-harm is meaningfully better covered than third-person family-member self-harm.
A "hybrid bridge" routes complex multi-person deliverable turns to a bigger model. Built and honestly evaluated — and found net-negative: both return paths were skipping memory write-back, causing wrong-fact errors on the next turn. Measured trust scores: ~9.85/10 bridge-off vs. ~3.5/10 bridge-on. A feature that makes the product dramatically worse when enabled is a finding worth respecting. DISABLE_BRIDGE has never been explicitly set in production — flagged in July as needing a decision, still unset/ambiguous.
The core engine has been deliberately frozen since July 15, 2026 — zero engine commits between then and the August 16 audit, by stated policy: "stabilize before building the next layer." What kept moving: the gold-standard synthetic test dataset grew from 547 to 650+ rows across 18 simulated families, plus health-check monitoring and watchdogs. The most recent multi-family synthetic diagnostic (49 production-faithful contexts, simulated depths of 28–42 days) found zero structural failures at full depth, while ~33% of turns still showed quality-level gaps (stale-turn answers, tone mismatch, canned phrasing) — real but non-structural, and explicitly a synthetic offline evaluation, not live multi-user traffic.
Both safety defects above were introduced or persisted under that freeze — which is exactly why the freeze-then-audit-then-fix sequence exists. Finding these two is the audit doing its job.
Deliberately the shortest section, for the most honest reason available: the family/multi-user mechanics are under-documented as a distinct architecture area in every source this guide draws on. What follows is confirmed; what's absent is flagged rather than filled with guesses.
Confirmed, engine side live: a family roster and exclusions system feeds the governance gate — the unknown-family-member detection and allergen/exclusion checks from Safety both draw on it. It's why Oi could verify "Emma" in Part 2. An emergency-family skill exists specifically for family-crisis scenarios (with the routing caveat from Safety). The prototype-era skill catalog includes family-oriented skills by name: family-announce, family-bonding, family-film, family-comms.
Confirmed, platform side live: the OiMy-Platform web surface is explicitly multi-tenant and family-shaped — one isolated OpenClaw agent container per family, with the Tradclaw-inspired onboarding interview generating household-level USER.md/HEARTBEAT.md files (full lineage below).
The known gap, stated plainly: a proper deep-dive into multi-user mechanics — how roster membership is established and mutated, how permissions work between family members, how the engine attributes messages within a household — would require a targeted read of archetypes/action.py and the profile/roster code beyond what the compiled research covers. This guide says "real but not fully mapped" rather than inventing a tidy answer.
OiMy did not appear fully formed. Every sibling directory examined ties back to the product in a real, documented way — production, prototype, or training pipeline.
Claire Vo open-sourced Tradclaw (github.com/clairevo/tradclaw), a household-AI-assistant scaffold. Directly documented in OiMy-Platform/docs/OIMY_PLATFORM.md (originally titled TRADCLAW_PLATFORM.md), crediting it explicitly: "Adapted from Claire Vo's Tradclaw by the OiMy team." Four patterns were adapted for OiMy's multi-tenant Docker context — Tradclaw is single-user; OiMy-Platform is one container per family: the Skill Submission API gained an admin-review step; five heartbeat templates (Family Hub, Solopreneur, Home Manager, Health Focus, Minimal) ship pre-built; the five-question onboarding interview writes generated USER.md/HEARTBEAT.md into each family's container; the morning-brief cron became timezone-aware. This lineage lives entirely on the platform surface — none of it touches the Telegram engine.
1. The actual OiMy chat-UI prototype — a React web app ("ai-chat-app") with Clerk auth and framer-motion/lottie animations. Currently being repurposed into a testing client pointed at the dev engine.
2. A licensed content export from a real third-party self-improvement app called "Fabulous" (119 journeys, 501 chapters, 1,944 content steps, 386 habits, 331 tips, 114 audio trainings, 316 coaching entries), triaged and mined with founder-cleared licensing into OiMy's coaching-anchor file — which grew from 15 to 29 frameworks partly through this process. The refinement method: strip borrowed authority and citations, re-address broadcast copy as one-to-one conversation, de-moralize (no "clean" vs. "dirty" food, no willpower framing), convert commands into offers, replace completion-screen fanfare with specific noticing.
An on-device intent-routing prototype: Gemma-4-E4B as the router, 81 skills, 6 archetypes, SQLite-only, explicitly "isolated from the production OiMy codebase" per its own README, May 2026 timeline. The project's July 15 brief labels it plainly: "an earlier engine iteration, since superseded… legacy reference only, don't treat as current."
The best one-line reading: the archetype pattern and the five-layer memory design genuinely carried forward into production; the on-device-Gemma-router thesis that motivated building it did not. Production's live chat path is all cloud — while the on-device idea didn't die, it continued as the separate fine-tuning research track (below).
The fine-tuning pipelines and training data behind the three-round Gemma 4 12B LoRA effort measured in Model routing. Trained against real OiMy production behavior categories and real production code paths, gated on human review before any production consideration. It reads from production reality and writes back nothing — yet.
Listed only for completeness: the multi-tenant Next.js web app and control plane from Part 1's Surface 2, connected to the engine surface via shared Postgres-level user/session data.
Also honestly noted: oimy-ml-artifacts's internals were not deep-dived beyond confirming its role as the LoRA/dataset home, and the landing-v*/scratch folders are prior marketing-site iterations — out of scope. Nothing in the directories examined was unrelated to OiMy.
Here's a fact this guide could have quietly smoothed over, and won't: the files literally named ARCHITECTURE.md, BACKGROUND.md, and README.md at the top level of the oimy-engine repo do not describe the production system. They are byte-identical — confirmed via md5sum — to the docs in the sibling oimy-skill-engine prototype. They describe the May-2026 on-device Gemma-4-E4B router prototype, complete with its own admission that it's "isolated from the production OiMy codebase." Anyone who opens the production repo and reads its front-door README is reading a museum placard hung on the wrong exhibit.
The more accurate description lives one directory down, in oimy-engine/docs/ARCHITECTURE.md — a different file, itself still confusingly titled "OiMy Skill Engine — Architecture" — whose file map matches the real files the 2026-08-16 audit actually cites in live production.
It would have been easy to fix the mislabel quietly and pretend the docs were always in sync. Instead, notice what this project actually did, in order: the July 15 architecture brief explicitly labeled the prototype "superseded… legacy reference only, don't treat as current." Then it commissioned an independent audit that trusted no document and traced every claim to a file and line — which is the only reason this guide can tell you, with citations, which memory layers are live, which env var is never read, and which two safety paths are broken. Then it asked for this explainer with one standing instruction: an accurate "we don't fully know X" beats a confident fabrication every time.
Stale docs are not the interesting fact — every fast-moving project has them. The interesting fact is the response: this project tells the truth about itself, on the record, in dated documents. That's why this guide can carry status tags like gated off and never shipped without hedging, and why you — the developer reading this — can trust the live tags precisely because the planned tags exist. If you build on OiMy, you inherit that norm. It's the most valuable thing in the repo, and it doesn't appear in any file map.
| Subsystem | Status |
|---|---|
| Telegram pipeline (Hermes → router → archetypes → governance → write-back) | live |
| Web app: OiMy-Platform + per-family OpenClaw containers | live |
| Memory L1 entity graph, L2 vector memory | live bugs |
| Memory L3 deep profile inference (Hughes / Maslow / rapport / beliefs) | gated off — Tier-1 default |
| Memory L4 journey tracker, L5 post-session pipeline | planned |
| Skills engine — 6 archetypes, ~80–90 YAML-manifest skills | live bugs |
| Light Hermes gateway (Telegram; iMessage reportedly running) | live per July-15 brief |
| Heavy Hermes | designed, deliberately deferred |
| Cloud model stack (Grok 4.3 / GPT-5.4-mini / DeepSeek / Sonnet 4.6 / Haiku 4.5) | live bugs |
| Local Gemma 4 12B LoRA track (v2: 7.10 vs 6.27; v3.1: 6.97; Grok: 7.40) | research track — never served a user |
| Safety: governance gate, roster/ledger checks, clinical filters | live |
| Safety defect: request_ctx NameError breaks crisis fallback | live bug, confirmed + reproduced |
| Safety defect: third-person self-harm misses synchronous crisis path | live gap, confirmed + reproduced |
| Hybrid bridge | built; net-negative in eval |
| Family / multi-user mechanics | real; under-documented — known gap |
| Core engine change freeze (since 2026-07-15) | deliberate policy |
| Usage / scale numbers (users, volume, uptime) | unknown — not in any audited source; none invented here |
How this guide was built. Research grounded in the project's dated internal docs (architecture briefs, integration reports, internal design decisions) and a rigorous, file:line-cited independent audit of live production performed 2026-08-16. Content written by Fable 5. Assembled, diagrammed, and deployed by Claude Code. Diagrams are hand-authored inline SVG — no AI image generation was available in this environment, so nothing here is a generated illustration; every diagram is a real explanatory drawing of an actual mechanism.
This is a living reference for a fast-moving system. Treat every citation as a pointer back to the source, and treat every "we don't know" as an invitation to go find out.