# OiMy — Next Tasks
## Post-Deploy Quality Sprint

**Date:** June 24, 2026  
**Context:** Phases 0–4 of the companion quality plan have been implemented and deployed. Engine is live on Hetzner at `116.203.107.13`, 89 skills healthy. This file covers the next batch of work.

---

## Current State (as of June 24 EOD)

**Already live in production:**
- ✅ `gemma4_12b_companion_v2.txt` — 17 behavioral fixes (warmth preserved, register matching, BE_PRESENT mode, no deferral, banned clinical words, etc.)
- ✅ `_classify_turn_mode()` — deterministic move selection: JUST_RESPOND / BE_PRESENT / PLAN_FORGE / ACKNOWLEDGE_AND_HELP
- ✅ `_extract_session_facts()` — names/ages/constraints pinned to system prompt each turn
- ✅ `_track_session_threads()` + `_get_thread_hints()` — named people + events tracked, re-injected as open threads
- ✅ `_get_trust_level()` — trust-gated directiveness ceiling based on session count + disclosure depth
- ✅ All three companion call paths (Ollama, OpenRouter, DeepSeek) receive `system_additions`

**Key files:**
- Engine: `/opt/oimy-engine/api.py`
- Companion prompt v2: `/opt/oimy-engine/prompts/gemma4_12b_companion_v2.txt`
- Rhythm runtime: `/opt/oimy-engine/archetypes/rhythm/rhythm_runtime.py`
- Rhythm delivery: `/opt/oimy-engine/archetypes/proactive_outreach.py` (has `send_outreach_message()` with direct Telegram API)
- Rhythm cron: `/opt/oimy-engine/scripts/rhythm_all_users.sh` → hits `/rhythm/morning` and `/rhythm/evening` on `localhost:8080`

---

## Task 1 — 5-Session NPS Spot Check (1 day)

**Why:** Confirm the tone_mismatch and unsolicited_advice counts drop with V2 prompt + move classifier active. No RunPod needed — runs against production on Hetzner.

**What to run:** Modified version of the eval script, 5 sessions only, against `http://localhost:8080`, using Grok 4.3 as user simulator and GPT-5.5 as QA judge.

**Key dimensions to check:**
- `tone_mismatch` count (was 68/360 turns = 18.9% — should be < 10%)
- `unsolicited_advice` count (was 27 — should be < 10)
- `trust_signals` score (was 4.02 — should hold or improve)

**Script reference:** Base on `/opt/oimy-engine/scripts/eval_gemma4_multisim.py`, but:
- `OIMY_API_URL = "http://localhost:8080"` (no RunPod)
- 5 sessions only (personas 1, 3, 5, 26-casual, 28-deep-disclosure)
- QA rubric already has the expanded dimensions from yesterday's eval
- Save report to `/opt/oimy-engine/docs/synthetic-user-testing/reports/spot5-postdeploy-YYYYMMDD.md`

**Get a token first:**
```python
import requests
token = requests.get("http://localhost:8080/token?user_id=spot5_eval_001").json()["token"]
```

**Acceptance:** Run completes, report generated. Compare governance_flags totals vs June 24 eval baseline.

---

## Task 2 — Quick-Action Buttons on Rhythm Messages (2 days)

**Why:** From the article analysis — parents should be able to respond to OiMy rhythm messages with a single tap, not compose a reply. "Rectangle problem": picking up phone to type breaks presence with kids.

**What it looks like:**
```
🌅 Good morning, Priya. How did yesterday's morning routine go?

  [Still figuring it out]  [Actually went well!]  [Different thing today]
```

Each button tap sends a pre-composed message into the chat, setting the session context immediately. No typing required.

### Architecture

The rhythm endpoint (`/rhythm/morning`, `/rhythm/evening`) in `api.py` calls `RhythmRuntime.run_morning_brief()` which returns:
```json
{"message": "text of morning brief", ...}
```

The endpoint does `self._json_response(result)`. The OpenClaw container skill (`oimy-forward`) calls this endpoint and sends the result text to Telegram.

**Two-part change:**

**Part A — Add `reply_markup` to rhythm response**

In `archetypes/rhythm/rhythm_runtime.py`, modify `run_morning_brief()` to also return `reply_markup`:

```python
def run_morning_brief(self, user_id, scope="individual", member_scope=None, today=None):
    # ... existing logic ...
    brief_text = brief.render_text()
    
    # Build context-aware quick-reply buttons
    buttons = self._build_rhythm_buttons(user_id, brief_text, period="morning")
    
    result = {"message": brief_text}
    if buttons:
        result["reply_markup"] = {
            "inline_keyboard": [buttons]  # single row of 2-3 buttons
        }
    return result

def _build_rhythm_buttons(self, user_id: str, brief_text: str, period: str) -> list:
    """
    Return 2-3 inline buttons appropriate for this rhythm message.
    Buttons send a pre-composed message when tapped.
    
    Rules:
    - Morning: always include a "Something else on my mind" option
    - If brief mentions a specific topic (e.g. "morning routine"), include 
      a topic-specific option
    - Max 3 buttons, each label ≤ 20 chars
    - callback_data = the message to send when tapped (OiMy receives it as 
      a normal chat message)
    """
    import re
    
    # Default morning buttons
    if period == "morning":
        buttons = [
            {"text": "Going well ✓", "callback_data": "Morning is going well today, thanks for checking in"},
            {"text": "Still tricky", "callback_data": "Still working through some things this morning"},
            {"text": "Something else", "callback_data": "I have something different on my mind today"},
        ]
    else:  # evening
        buttons = [
            {"text": "Good day ✓", "callback_data": "Today was a good one"},
            {"text": "Rough one", "callback_data": "Today was pretty rough honestly"},
            {"text": "Just checking in", "callback_data": "Just checking in, nothing major"},
        ]
    
    # Topic-specific overrides: if brief mentions a recent topic, add a relevant button
    # e.g. if the brief mentions "Arjun's tantrums" → add "Update on Arjun"
    name_match = re.search(r"\b([A-Z][a-z]+)'s\b|\bwith ([A-Z][a-z]+)\b", brief_text)
    if name_match:
        name = name_match.group(1) or name_match.group(2)
        buttons = [
            {"text": f"Update on {name}", "callback_data": f"Update on {name} — "},
        ] + buttons[:2]  # replace last button, keep first two
    
    return buttons[:3]  # max 3
```

**Part B — Deliver `reply_markup` to Telegram**

The rhythm endpoint in `api.py` currently calls `self._json_response(result)` and the container skill delivers only the `message` field. 

**Check the `oimy-forward` skill** (`/opt/oimy/skill-templates/oimy-forward/SKILL.md`) to see if it forwards `reply_markup` from engine responses. If not, two options:

**Option 1 (Preferred):** Have the rhythm endpoint send the Telegram message directly (like `send_outreach_message()` in `proactive_outreach.py`), including `reply_markup`:

In `api.py` rhythm handler, after getting `result`:
```python
result = rhythm.run_morning_brief(...)

# If reply_markup present: send directly via Telegram Bot API, return confirmation only
if result.get("reply_markup"):
    bot_token = os.environ.get("TELEGRAM_BOT_TOKEN", "")
    chat_id = user_id  # user_id is the Telegram chat ID for rhythm
    if bot_token and chat_id:
        _send_telegram_with_buttons(bot_token, chat_id, result["message"], result["reply_markup"])
        self._json_response({"status": "sent", "message": result["message"]})
        return

self._json_response(result)
```

Add helper function:
```python
def _send_telegram_with_buttons(bot_token: str, chat_id: str, text: str, reply_markup: dict):
    import json as _json, urllib.request as _ureq
    payload = json.dumps({
        "chat_id": chat_id,
        "text": text,
        "reply_markup": reply_markup,
        "parse_mode": "HTML"
    }).encode()
    req = urllib.request.Request(
        f"https://api.telegram.org/bot{bot_token}/sendMessage",
        data=payload,
        headers={"Content-Type": "application/json"}
    )
    try:
        urllib.request.urlopen(req, timeout=10)
    except Exception as e:
        print(f"  [rhythm buttons] Telegram send failed: {e}")
```

**Option 2:** Modify the `oimy-forward` skill to forward `reply_markup` if present in engine response.

Check which option is cleaner based on how `oimy-forward` skill works.

**Handling button taps:**  
When a user taps a button, Telegram sends a `callback_query` to the bot. The `callback_data` value becomes the message text. No special handling needed on the engine side — the container should already convert `callback_query` updates into normal chat messages. Verify this is the case in the OpenClaw container config.

**Get the Telegram Bot Token:**
```bash
grep TELEGRAM_BOT_TOKEN /opt/oimy-engine/.env /etc/systemd/system/oimy-engine.service 2>/dev/null
```
If not in env, check OpenClaw config: `openclaw config get channels.telegram.botToken`

### Acceptance
- [ ] Morning rhythm message arrives with 3 inline buttons
- [ ] Tapping a button sends that text as a chat message
- [ ] Engine handles the button message as a normal chat turn (JUST_RESPOND mode expected for "Going well ✓")
- [ ] Evening message also has appropriate buttons

---

## Task 3 — Voice Note → TTS Reply (1 day)

**Why:** From article analysis — "raise wrist, speak, lower wrist." When a parent sends a voice note, OiMy should reply with audio so the whole interaction is hands-free. No screen needed.

**What it looks like:**
- Parent speaks a voice note via Telegram (or AirPods via Siri)
- OiMy processes the transcription (already happens via whisper/transcription)
- OiMy replies with a TTS audio message (ElevenLabs)

**Current state:**
- `EL_VOICE_AGENT_ID` is set in `/opt/oimy-engine/.env` (ElevenLabs voice agent configured for voice calls)
- Voice note transcription already works (engine receives transcribed text as `user_input`)
- Missing: detecting that input came from voice → replying with TTS

### Architecture

**Part A — Detect voice input**

When a Telegram voice note is sent, OpenClaw transcribes it and routes to the engine as a normal chat message. The engine needs to know the input was voice so it can return TTS audio.

Add a flag to the chat request: `"input_type": "voice"` (the OpenClaw Telegram plugin should already have this metadata).

Check in `api.py` chat handler:
```python
# Check input_type in the POST body
input_type = data.get("input_type", "text")  # "voice" or "text"
```

**Part B — Generate TTS reply when input was voice**

After the engine generates a reply, if `input_type == "voice"`, call ElevenLabs TTS:

```python
def _tts_reply(self, text: str, user_id: str) -> Optional[bytes]:
    """
    Convert reply text to speech using ElevenLabs.
    Returns audio bytes (mp3) or None on failure.
    
    Uses ElevenLabs text-to-speech API directly (not the voice agent/streaming endpoint).
    Voice: configured via EL_TTS_VOICE_ID env var (get from ElevenLabs dashboard).
    """
    import os, urllib.request, json
    
    el_api_key = os.environ.get("ELEVENLABS_API_KEY", "")
    el_voice_id = os.environ.get("EL_TTS_VOICE_ID", "")  # standard TTS voice, not the agent
    
    if not el_api_key or not el_voice_id:
        return None
    
    # Trim text to reasonable TTS length (max 500 chars)
    tts_text = text[:500].strip()
    
    payload = json.dumps({
        "text": tts_text,
        "model_id": "eleven_turbo_v2_5",  # fastest, lowest latency
        "voice_settings": {"stability": 0.5, "similarity_boost": 0.75}
    }).encode()
    
    req = urllib.request.Request(
        f"https://api.elevenlabs.io/v1/text-to-speech/{el_voice_id}",
        data=payload,
        headers={
            "xi-api-key": el_api_key,
            "Content-Type": "application/json",
            "Accept": "audio/mpeg"
        }
    )
    try:
        resp = urllib.request.urlopen(req, timeout=15)
        return resp.read()
    except Exception as e:
        print(f"  [tts] ElevenLabs failed: {e}")
        return None
```

In the chat handler response, add audio if input was voice:
```python
if input_type == "voice":
    audio_bytes = self._tts_reply(reply_text, user_id)
    if audio_bytes:
        # Return audio as base64 in response for OpenClaw to send as voice message
        import base64
        response["audio_reply"] = base64.b64encode(audio_bytes).decode("utf-8")
        response["audio_mime"] = "audio/mpeg"
```

**Part C — Check if OpenClaw handles audio_reply**

Check if the OpenClaw channel plugin handles `audio_reply` in engine responses and sends it as a Telegram voice message. If not, the engine can send the voice message directly via Telegram Bot API:

```python
# Send audio directly to Telegram
def _send_telegram_audio(bot_token: str, chat_id: str, audio_bytes: bytes):
    import urllib.request
    # Use multipart/form-data to send voice message
    boundary = "boundary"
    body = (
        f"--{boundary}\r\n"
        f'Content-Disposition: form-data; name="chat_id"\r\n\r\n{chat_id}\r\n'
        f"--{boundary}\r\n"
        f'Content-Disposition: form-data; name="voice"; filename="reply.mp3"\r\n'
        f"Content-Type: audio/mpeg\r\n\r\n"
    ).encode() + audio_bytes + f"\r\n--{boundary}--\r\n".encode()
    
    req = urllib.request.Request(
        f"https://api.telegram.org/bot{bot_token}/sendVoice",
        data=body,
        headers={"Content-Type": f"multipart/form-data; boundary={boundary}"}
    )
    urllib.request.urlopen(req, timeout=20)
```

**Required env vars to add to `/opt/oimy-engine/.env`:**
```
EL_TTS_VOICE_ID=<voice_id_from_elevenlabs_dashboard>
ELEVENLABS_API_KEY=<api_key>
```

Check if `ELEVENLABS_API_KEY` is already set:
```bash
grep ELEVEN /opt/oimy-engine/.env
```

### Acceptance
- [ ] Send a voice note to OiMy via Telegram
- [ ] Engine generates a reply
- [ ] Reply arrives as a Telegram voice message (not text)
- [ ] Text fallback still works if TTS fails

---

## Task 4 — Siri Shortcut for Watch Dictation (half day, no code)

**Why:** Zero new infrastructure. iMessage already routes to OiMy. An Apple Watch shortcut lets parents send voice commands to OiMy hands-free while present with their kids.

**What to build:** A shareable Siri Shortcut URL + setup guide hosted on CDN.

### The Shortcut

Build in Shortcuts app on iPhone:
1. Action: **Dictate Text** (speech → text)
2. Action: **Send Message** → iMessage → OiMy iMessage contact → `[Dictated Text]`

Export as a shareable link: `https://www.icloud.com/shortcuts/...`

### The Setup Guide

Create `/var/www/models/docs/oimy-siri-shortcut.html` with:
- One-tap "Add to Siri" button linking to the shortcut URL
- 3-step setup (30 seconds to install)
- Short explainer: "Raise wrist → 'Hey Siri, Ask OiMy' → speak → done"
- Works with iPhone, Apple Watch, AirPods
- QR code linking to the page (for sharing in onboarding)

**File to create:** `oimy-siri-shortcut.html` on CDN at `https://cdn.oimyai.com/docs/oimy-siri-shortcut.html`

**Note:** The iMessage channel (`/opt/oimy/photon/`) must be running for this to work. Check: `ps aux | grep agent.mjs`

---

## Task 5 — Context-Triggered Proactive Nudges (1 week)

**Why:** Highest-leverage ambient intelligence item from the article. "It shows up when it matters" — not just on a static morning/evening cron, but when it's actually relevant.

**What it does:** After each conversation, scan for forward-looking signals (dates, events, "I'm going to try X this week"). Automatically create a scheduled nudge for the right time.

**Examples:**
- User says "Arjun has a school presentation Thursday" → schedule a nudge for Wednesday evening: *"Arjun's presentation is tomorrow — anything you want to run through together tonight?"*
- User says "we're trying a new bedtime routine starting Monday" → schedule a Monday evening nudge: *"First day of the new bedtime routine — how did it go?"*
- User commits to something: "I'll try the 10-minute warning tomorrow" → schedule a next-day nudge: *"How did the 10-minute warning go this morning?"*

### Architecture

**Part A — Signal extractor** (runs after each chat_light turn)

```python
def _extract_forward_signals(self, user_input: str, conversation_history: list) -> list:
    """
    Extract forward-looking signals from conversation.
    Returns list of {type, content, suggested_nudge_time, suggested_nudge_text}.
    
    Types:
    - upcoming_event: "presentation Thursday", "appointment next week"  
    - commitment: "I'm going to try X", "we'll start Y on Monday"
    - plan_checkin: existing plan from PLAN_FORGE that needs a follow-up
    """
    import re
    signals = []
    text = user_input.lower()
    
    # Upcoming events with dates
    day_words = r'(monday|tuesday|wednesday|thursday|friday|saturday|sunday|tomorrow|next week|this weekend)'
    event_words = r'(presentation|appointment|meeting|game|recital|play|surgery|interview|checkup|test|exam)'
    event_match = re.search(f'{event_words}.{{0,30}}{day_words}|{day_words}.{{0,30}}{event_words}', text)
    if event_match:
        signals.append({
            "type": "upcoming_event",
            "content": event_match.group(0),
            "raw_input": user_input
        })
    
    # Commitments
    commitment_match = re.search(
        r"\b(i('ll| will) (try|start|do|use|attempt)|we('re| are) going to|starting (monday|tomorrow|next week)|going to try)\b",
        text
    )
    if commitment_match:
        signals.append({
            "type": "commitment", 
            "content": user_input,
            "raw_input": user_input
        })
    
    return signals
```

**Part B — Nudge scheduler** (runs after signal extraction)

```python
def _schedule_nudge_from_signal(self, user_id: str, signal: dict):
    """
    Create a scheduled nudge entry in the DB for this signal.
    The nudge gets delivered by the rhythm cron at the right time.
    
    Table: proactive_nudges
    {user_id, trigger_type, trigger_content, nudge_text, send_after, status, created_at}
    """
    import sqlite3, json
    from datetime import datetime, timedelta
    
    # Call the LLM to generate the nudge text (one quick call, low cost)
    # Only do this for high-confidence signals
    nudge_text = self._generate_nudge_text(user_id, signal)
    if not nudge_text:
        return
    
    # Determine send_after time
    send_after = self._estimate_nudge_time(signal)
    
    db_path = "/opt/oimy-engine/data/oimy.db"
    conn = sqlite3.connect(db_path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS proactive_nudges (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            user_id TEXT NOT NULL,
            trigger_type TEXT,
            trigger_content TEXT,
            nudge_text TEXT NOT NULL,
            send_after TEXT NOT NULL,
            status TEXT DEFAULT 'pending',
            created_at TEXT DEFAULT CURRENT_TIMESTAMP
        )
    """)
    conn.execute(
        "INSERT INTO proactive_nudges (user_id, trigger_type, trigger_content, nudge_text, send_after) VALUES (?,?,?,?,?)",
        (user_id, signal["type"], signal["content"][:200], nudge_text, send_after.isoformat())
    )
    conn.commit()
    conn.close()
```

**Part C — Delivery** 

Extend `rhythm_all_users.sh` to also check and deliver pending proactive nudges at each 5-minute tick:

```bash
# After sending rhythm messages, check for pending nudges
python3 /opt/oimy-engine/scripts/deliver_proactive_nudges.py
```

### Acceptance
- [ ] "presentation Thursday" → nudge fires Wednesday evening
- [ ] Commitment "I'll try X" → follow-up nudge fires next day
- [ ] Nudge not sent if user already messaged OiMy that day (already engaged)
- [ ] Nudge is warm and contextual, not generic

---

## Environment Variables to Add

```bash
# Add to /opt/oimy-engine/.env and /etc/systemd/system/oimy-engine.service

# Task 3: ElevenLabs TTS
ELEVENLABS_API_KEY=<from ElevenLabs dashboard>
EL_TTS_VOICE_ID=<voice_id from ElevenLabs dashboard — warm female voice>

# Task 2: Telegram Bot Token for direct button delivery
# Already should exist — confirm with:
# grep TELEGRAM_BOT_TOKEN /opt/oimy-engine/.env
```

After adding env vars, reload and restart:
```bash
systemctl daemon-reload
systemctl restart oimy-engine.service
```

---

## Priority Order

| Task | Est. | Impact | Do first? |
|------|------|--------|-----------|
| Task 1 — NPS spot check | 1 day | Confirms the work done today actually moved the needle | Yes — do first |
| Task 2 — Rhythm buttons | 2 days | Direct UX improvement, immediate user impact | Yes |
| Task 3 — Voice TTS | 1 day | High delight, enables hands-free use | Yes |
| Task 4 — Siri Shortcut | 0.5 day | Zero code, high marketing value | Whenever |
| Task 5 — Context nudges | 1 week | Highest long-term leverage | After 1-3 done |

---

## Task 6 — Mac App Build + Test (2-3 weeks)

**Full plan:** `docs/OIMY-MAC-APP-PLAN.md` in this repo (OiMy-Platform on GitHub).

**Summary of what to build:**
- Self-contained `.dmg` installer: Ollama (bundled, pinned version) + Gemma 4 12B QAT q4 + OiMy Engine (Python) + Whisper STT + Cloudflare tunnel
- Swift menu bar app: status icon, start/stop, opens chat UI in WKWebView
- First-run wizard: model download (~7 GB), API key setup, optional tunnel config
- Voice mode: mic button → Whisper STT → OiMy → Kokoro/ElevenLabs TTS → audio playback
- Target hardware: Apple Silicon M1+ 16 GB minimum, M4 24 GB recommended

**Model config for Mac:**
```
COMPANION_IS_OLLAMA=1
COMPANION_API_URL=http://localhost:11434/api/chat
COACH_IS_OLLAMA=1
COACH_API_URL=http://localhost:11434/api/chat
COMPANION_DEEP_THINK=0
OLLAMA_MODEL=gemma4-qat-q4
FORCE_LOCAL_MODEL=1
```
Both companion AND BCT paths use 16k context, think=False for conv / think=True for BCT.

**Companion prompt:** V2 (`prompts/gemma4_12b_companion_v2.txt`) — this must be the active prompt in the Mac app bundle. The code already loads v2 first if present.

**Performance targets:**
- First turn latency (warm): < 8 seconds on M2 16 GB
- Sustained latency: < 12 seconds
- Memory: < 11 GB total (Ollama + engine)
- NPS: ≥ 7.0 on 5-session spot check

**Testing sequence:**
1. Smoke test (engine health + routing + model inference)
2. 5-session NPS check: `eval_gemma4_multisim.py` against `localhost:8080`, 5 sessions, Grok 4.3 sim, GPT-5.5 judge
3. Latency benchmark: 10 turns, measure P50/P95
4. Test on M2 16 GB (tight) + M4 24 GB (comfortable)

**Reference files:**
- Mac app plan: `docs/OIMY-MAC-APP-PLAN.md` (GitHub)
- Engine state: `docs/OIMY-ENGINE-STATE-2026-06-24.md` (GitHub)
- UX design: https://cdn.oimyai.com/docs/oimy-mac-app-ux-v1.html
- Gemma 4 QAT model: `hf.co/google/gemma-4-12B-it-qat-q4_0-gguf`

**Phase order:**
1. Phase 1: Core bundle (Swift launcher + Ollama + engine) — 1 week
2. Phase 2: Voice STT + TTS — 3-4 days
3. Phase 3: Cloudflare tunnel — 2 days
4. Phase 4: Testing on real hardware
5. Phase 5: Packaging + DMG