Full end-to-end architecture — local and cloud models, the hybrid routing between them, recent fine-tuning and gold-standard dataset work, every layer a message passes through, and an honest read on what's ready for outside testing today versus what still needs building.
The short version: real users today talk to OiMy running on a cloud model (Grok 4.3), through a proven, working path. A local model (fine-tuned three times now) exists and is genuinely good in testing, but isn't serving real users yet. A sturdier next-generation core engine exists in code and is running, but only for one test account so far. A bigger upgrade to the chat layer ("Heavy Hermes") is designed but not built. A friendly "watch OiMy think" view for testers doesn't exist yet — that's new work, in progress now.
Everything below is cited against real files, commits, eval results, and live server checks — not against older planning notes, some of which had drifted from what's actually running.
Someone texts OiMy on Telegram (or, soon, iMessage). Here's what really happens between that message and a reply landing back in their chat:
/chat endpoint receives the forwarded message and figures out the session boundary (is this a continuation of the last conversation, or a fresh one after a 30‑minute gap?).OiMy isn't running on one single model — there are two tracks, and a (currently paused) bridge meant to connect them:
| Cloud (serving real users) | Local (fine-tuning & eval track) |
|---|---|
| Live, production Grok 4.3 is the actual companion model real conversations run on today. Router = GPT‑5.4‑mini, deeper-reasoning = Claude Opus 4.8, tall/escalation-reserve = Kimi K2.6. | Eval-only, not serving users Gemma 4, 12B, QAT (a compact model that can run outside the big cloud providers), run on rented GPU for training and evaluation. Fine-tuned three times so far (below) with real, measured quality gains — but it has never been the model a real user's message actually goes to. |
The hybrid idea: use the local model for routine, everyday turns (cheap, fast, private, good enough), and only "bridge" a turn to a bigger cloud model when it's genuinely complex (a multi-step logistics ask, a high-stakes emotional moment). This bridge was actually built and tested — and the honest result was mixed enough that it's not something we can currently point to as "done":
Known issue, needs a decision: the hybrid bridge (routes complex "deliverable" turns with more than 3 people involved to a bigger model) was tested and found net negative as built — both of its return paths skip writing the result back into the conversation's memory, so the next message gets basic facts wrong (like which kid something was about). In eval, that dropped a trust score from ~9.85/10 with the bridge off to ~3.5/10 with it on. As far as we can tell, production still has this bridge in a not-fully-off state (DISABLE_BRIDGE was never explicitly set) — worth explicitly confirming/fixing rather than assuming it's harmless.
All of this work is on the local Gemma 4 model only — it has never touched the production Grok 4.3 model real users talk to.
| Round | What changed | Result |
|---|---|---|
| v1 (pilot) | First proof that fine-tuning actually changes behavior at all (~$7 of training). | Worked, but overcorrected into overly terse replies. Not shipped. |
| v2 | 19,074 real training examples. | Full 30-session test: 7.1/10, vs. 6.27 baseline (+0.83) — a real, meaningful improvement. One known gap: sometimes under-delivers on multi-part asks. |
| v3.1 | Added 3,400 "show your reasoning" examples + 733 fixes, 23,866 total. | Beat the un-tuned model more decisively than v2 did, and fixed the multi-part-ask gap — but landed at essentially the same overall score as v2 (6.97, a wash), while introducing a small new quirk (occasionally too brief on short exchanges). |
All training data and results are backed up in the oimy-ml-artifacts repo. The next planned step — converting the fine-tuned model to run faster on Apple hardware — hasn't happened yet; it's stuck behind a currently-unavailable dev machine connection.
To have real, high-quality examples to train and test against, we built a hand-curated set of realistic family conversations — 18 different families, spanning story-arcs as long as 42 simulated days, 547+ individual conversation turns as of the last count. These were authored using GPT‑5.6 (to plan realistic scenario arcs) plus an independent AI reviewer voicing the actual responses, so the "gold" answers aren't just OiMy grading its own homework.
Honest quality read: the content itself is good enough to treat as the bar to hit. The production line to build a lot more of it fast is not ready yet — some quality-control steps that should be automatic were still manual as of the last review, a planned check for cultural/language authenticity hadn't been run on 3 of the families, and one family's data had an internal contradiction that needed fixing. An earlier idea to scale this to "hundreds of families" quickly was explicitly rejected as unrealistic without a real quality drop — more families, done right, takes real time per family.
Putting sections 01–04 together, here is the complete stack a message passes through:
| Light Hermes | Heavy Hermes |
|---|---|
| Live today A thin router. One job: decide "is this for OiMy" and forward it. No persona, no memory, no personality of its own. | Designed, not built A fuller adaptive persona layered onto the gateway itself, deliberately fed OiMy's real knowledge before every reply (a direct code hook, not hoping the model remembers to ask) — plus wired for other channels like iMessage. |
Why the split exists: the vendor gateway tool has a hard 20,000‑character cap on how much "personality" text it can carry — and OiMy's real coaching/knowledge file is over 6x that size. Light Hermes sidesteps the problem by carrying none of that weight. Heavy Hermes solves it properly, by injecting exactly what's needed per-turn through code rather than stuffing it all into one persona file.
Why Heavy Hermes isn't being built yet: when GPT‑5.6 and an independent Fable review compared notes on OiMy's real recent production bugs, only 1 in 6 traced back to an actual architecture gap — the rest were reliability issues in the core engine itself. Bharath's call, on that evidence, was to finish stabilizing the core engine first and build Heavy Hermes after, not alongside it.
"The app" (the web app at OiMy's main site, and the mobile app) is a separate presentation layer that talks to the same core engine through its own API — it doesn't go through Hermes at all. Hermes exists specifically for chat-style channels (Telegram, iMessage).
Dated 2026‑07‑11/12, this is the current, active body of work. It's phased on purpose — each phase ships to the dev environment first, then to a single real account (Bharath's) as a canary, before anyone considers wider rollout.
| Phase | What it does | Status |
|---|---|---|
| 0 — Auth hole | Closed a real bug where anyone could mint a valid login for any user, unauthenticated. | Fixed, dev + prod |
| 1 — Request isolation | Replaced shared per-thread state with an explicit per-request context, closing a class of bug where one user's data could leak into another's request under load. | Dev; 1‑account canary in prod |
| 2 — Durable delivery | A real ledger + outbox, so the system can prove a message was actually delivered instead of assuming it was. | Fully live, dev + prod |
| 3 — Context API | New /context endpoints that will let Hermes (light or heavy) pull OiMy's real knowledge instead of guessing at it. | Dev; 1‑account canary in prod |
| 4 — Bounded server | Replaced the original bare-bones server with a proper rate-limited, health-checked one. | Dev; 1‑account canary in prod |
| 5 — Monitoring & alerting | Would tell us automatically when something breaks, instead of finding out from a user. | Not started — next up |
| 6 — Data retention & auth hardening | Longer-term data policy and access control work. | Not started |
| 7 — Real testing program | This is, in effect, what Rohan's involvement is starting. | Not started as a formal program |
Important: every real user conversation today still runs on the original, pre-stabilization path. None of phases 1, 3, or 4 have moved beyond Bharath's own test account. Nothing is broken by this — it's deliberately cautious sequencing — but it means "the new engine" and "what real users experience" are currently two different things.
There's a real, substantial test worth knowing about here — with one important caveat on what kind of test it is. A diagnostic pass ran 49 production-faithful conversation contexts (real memory assembly, real knowledge retrieval, real conversation history depth) across the 18 gold-standard families, at simulated depths of up to 28–42 days into a family's story. Result: zero "the system falls apart at scale" failures — no repetition, no drift, no breakdown — even at full length and depth. About a third of turns still showed a "quality/behavior" gap (answering a question that was two turns old instead of the current one, a tone mismatch, canned-sounding coaching phrasing) — real things to keep improving, but not a structural failure.
The caveat: this was a synthetic, offline evaluation (simulated families and simulated days), not live traffic from real multi-user, multi-day usage. It's the closest and most rigorous thing we have to "does this hold together over time with more than one family," and it's a genuinely good result — just worth being precise that it's a rehearsal, not a live show, when describing it to others.
| Repo | What it is | Note for Rohan |
|---|---|---|
| oimy-engine | The real companion engine (api.py) — what users actually talk to. | Pushed to GitHub today — was previously 4 commits + uncommitted work stuck only on the server; now up to date, including this brief as a markdown file under docs/architecture/. |
| OiMy-Platform | The web app's Next.js API layer + skill/archetype definitions + architecture docs. | Up to date, reviewable now. |
| fantastic-app-provider | The actual polished chat UI ("ai-chat-app") — animations, chat rendering — that a new test client is being built from. | Being repurposed right now for a testing-only client (see below). |
| oimy-skill-engine | An earlier engine iteration, since superseded. | Legacy reference only — don't treat as current. |
| oimy-ml-artifacts | LoRA fine-tuning data + results for the local Gemma 4 model. | Reference for the fine-tuning section above. |
A dev instance of the new engine is live right now on the production server, health-checked and responding. It isn't directly reachable from the open internet on its own — it's meant to be reached through a per-person hostname, which needs a fresh, dedicated Telegram bot token to wire up properly (in progress).
A dedicated testing web client is being built right now, based on the existing polished chat UI, pointed at the dev engine rather than production, with message history/memory handled by the real backend exactly as it is for real users.
Today, the only thing close to this is a raw internal debug flag on the engine that dumps its internal field names straight through — the kind of thing meant for an engineer at 2am, not a friendly tester. Building a plain-language version of it — "Remembering your family" → "Deciding how deep to think" → "Double-checking the reply" → "Oi's answer," matching the layer names in section 05 — is underway as part of the new test client.
Compiled 2026‑07‑15 from live production checks, real code, and real eval results. Where something is a plan rather than a shipped fact, it's labeled that way above on purpose.