GPT-5.6 Independent Review — Hetzner Audit, Goal-2 Plan, and Production-Readiness

Date: 2026-07-11 Purpose: An independent, adversarial second opinion from a different AI lab's model (not Fable, not any Claude model), obtained by making real API calls to OpenRouter — not by having Claude paraphrase what it thinks GPT would say. Bharath's explicit ask was for a genuinely independent cross-check before approving Fable's audit + execution plan, and separately, an independent production-readiness assessment of OiMy as a whole.

Methods note

Call 1 — Verification of Fable's Goal-1 Audit + Goal-2 Execution Plan

(a) Is the container classification correct and appropriately conservative?

Verdict

Broadly conservative, but not fully proven and not accurately described as a classification of all 24 containers. I would not authorize deletion from this table. I would authorize only reversible stopping of the seven proposed test containers after immutable backups and stronger ownership/content checks.

The independent Docker evidence supports the operational split:

However, Docker state does not independently verify Fable’s identity mappings, workspace-content conclusions, or “active/working” labels. Those depend on Fable’s DB and filesystem investigation, which is described but not reproduced in the independent evidence.

Specific classification problems

1. “Real, active, verifiably working: 4” is an overstatement

The four containers are stable, not necessarily active or end-to-end working.

In particular:

Before calling any one of these “working,” I would require:

  1. Application-readiness evidence, not merely State.Status=running.
  2. Recent successful inbound and outbound message evidence.
  3. Successful context retrieval/writeback where applicable.
  4. Successful reminder/proactive-message execution if that feature is enabled.
  5. Confirmation that no Telegram polling/webhook conflict exists.

The correct rollup is closer to “4 process-stable containers, of which Fable believes 2 are real-user instances.”

2. The seven “safe to stop” labels are reasonable only if “stop” remains distinct from “delete”

The proposed seven are:

Fable did more than classify by name: it says it checked USER.md contents and found template-empty workspaces for the pool slots. That is directionally appropriate and more conservative than name-only classification.

But an empty USER.md does not prove there is no valuable state elsewhere. Before stopping—and definitely before later removing—each one should be checked for:

Then create a hashed archive plus docker inspect output and retain it under a documented restoration procedure. Stopping is reversible; docker rm, volume removal, workspace deletion, or DB-row deletion is not. The approval should state that none of the latter is authorized.

3. The “real/repair-first” bucket is appropriately conservative

Given the asymmetric harm Bharath specified, Fable made the right choice not to downgrade real-looking external accounts merely because they are broken.

The strongest case is oimy-pool-82c4, where Fable reports filled-in onboarding information. The Chris family containers also warrant preservation even without DB rows because they may be part of a real family deployment.

That said, those identity/content findings are not independently reproduced here. Before touching them, I would want the exact:

I would not require contacting every user merely to preserve or diagnostically clone the data, but restarting an exited family bot or changing its externally visible behavior should require Bharath’s approval and, where appropriate, user notification.

4. Fable did not actually classify all 24 Docker containers in its rollup

Its 4 + 11 + 7 categories total the 22 OpenClaw instances. The other two current Docker containers are:

They are infrastructure, but the plan should still state:

The “already gone” Chen and historical DB rows are not part of the current 24-container inventory.

5. The claim that a prior destructive Chen cleanup definitely happened is not established

Absence from Docker and /opt/oimy/containers, combined with DB status stopped, shows that the instances are absent now. It does not prove they once had months of data that someone destructively cleaned up. They could have failed before provisioning, been intentionally deprovisioned, or been stored elsewhere.

It is a valid reason to look for provisioning/deletion audit records and backups, but Fable states the destructive-cleanup inference too confidently.

6. “DB hygiene is zero risk” is wrong

Changing a DB instance from running to another state can affect management UI behavior, provisioning, reconciliation, or future cleanup automation. Historical rows should not be deleted casually. Reconciliation should:

Accuracy issue in the audit itself

The TL;DR restart range is wrong. The 18 broken containers include:

Thus the range is approximately 7,302–73,271, not 23,800–73,264. Also, the exited containers are not presently in a “continuous restart loop.” They accumulated many restarts and then stopped.

This does not invalidate the central fleet-failure finding, but it should be corrected before the document is used as an execution record.

(b) Is the OOM diagnosis correct?

What is confirmed

For oimy-pool-xoh3, the proximate failure is unquestionably a V8 JavaScript heap-limit abort:

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

It exits with code 1 and restarts approximately every 50–60 seconds.

The absence of kernel OOM records and OOMKilled=false is completely consistent with this. Those facts do not refute a memory-related failure; they show that V8 aborted before the kernel or cgroup OOM killer killed the process.

The independent evidence also confirms a 1 GiB container limit on three affected containers:

Therefore, raising the cgroup limit is a sensible experiment.

What is not confirmed

Fable has identified the failure mode, but not the root cause.

The evidence does not establish that “workspace/skill footprint grew beyond 1 GiB” is the reason. The supporting comparison is weak:

The heap reaches approximately 525 MB. That makes an explicit or runtime-derived ~512 MB heap ceiling particularly worth investigating.

Also, the independent evidence demonstrates the OOM signature for pool-xoh3, not all 18 broken instances. Fable reports seeing the same signature on sharma, but the exited Chris-family containers and every remaining instance need their own last-exit/log classification. It is unsafe to apply one remedy to all 18 solely from restart counts.

Will raising the memory limit fix it?

Possibly, but there are three materially different outcomes:

  1. Legitimate boot working set:
    At 2 GiB the process reaches readiness, memory plateaus, and remains stable. Raising the limit may be a valid fix.

  2. Explicit V8 cap:
    Raising only the cgroup may do nothing if --max-old-space-size remains explicitly set around 512 MB.

  3. Leak or runaway initialization:
    A larger limit merely delays the next crash and can make host impact worse.

Given 30 GB host RAM, raising eleven containers to 2–4 GiB creates substantial aggregate exposure. Cgroup limits are not preallocated, but if several processes use their new allowance simultaneously, the host could experience real memory pressure. Eleven 4 GiB ceilings already exceed physical RAM before accounting for stable containers and host services.

Fable’s proposed validation is insufficient

“Boot clean and no restart-count increment for 10 minutes” proves only that a larger limit delayed the failure beyond ten minutes. It does not distinguish a legitimate larger working set from a slow leak.

The diagnostic should be:

  1. Capture exact runtime configuration - Image ID/digest. - Entrypoint and command. - Node/OpenClaw versions. - NODE_OPTIONS. - v8.getHeapStatistics().heap_size_limit if feasible. - All memory/cgroup settings. - Comparison against a stable capped container and chris-b.

  2. Preserve and clone one test instance - Archive all mounts and writable-layer changes first. - Do not run the clone with the same Telegram token, webhook, cron jobs, or outbound credentials; otherwise two consumers can conflict or send duplicate messages.

  3. Run a controlled matrix - Same clone at 1 GiB. - Same clone at 2 GiB and/or 4 GiB. - Check behavior with and without any explicit heap setting. - Record RSS, V8 heap-used/heap-total, event-loop behavior, CPU, GC frequency, and time to readiness.

  4. Observe long enough to establish a plateau - At least several hours for the diagnostic canary, not ten minutes. - Then a longer soak before broad rollout.

  5. Test functionality, not just process survival - Readiness endpoint. - One isolated model/tool call. - Context load. - Persistence/writeback. - Clean restart and state recovery.

  6. Inspect each failure class - Capture exit code and final logs for every affected container. - Do not assume the exited Chris-family instances share the same cause.

The best description is: “V8 heap exhaustion is confirmed for at least one representative container; the role of the 1 GiB cap is plausible but unproven.”

(c) Is the Goal-2 sequencing and risk assessment sound?

Overall verdict

The high-level direction is sensible, especially putting the fleet failure ahead of architecture migration. But the execution sequence lacks a necessary data-preservation and capacity-safety gate, and several “risk to live traffic: none” claims are materially wrong.

Phase 0: needs a new Phase 0A before diagnosis or repair

Before changing memory limits, restarting exited containers, cloning workspaces, or changing restart policies:

  1. Freeze deletion and DB cleanup.
  2. Capture docker inspect, image digests, commands, environment metadata with secrets protected, mounts, restart policies, network mappings, and container diffs.
  3. Snapshot/archive every real or uncertain workspace and volume.
  4. Dump the relevant DB records.
  5. Hash the archives.
  6. Demonstrate restoration of one low-risk test instance.
  7. Establish host memory and concurrency limits for the repair campaign.

Only then should the memory experiment begin.

Phase 0 is not “risk to live traffic: none”

It touches containers tied to real users and a shared production host.

Risks include:

A safer rollout is:

  1. Back up everything.
  2. Stop or disable restart on verified-empty test containers after backup to reduce current load; this is reversible and does not require fixing them first.
  3. Diagnose one isolated clone.
  4. Add monitoring before broad repair.
  5. Repair one approved real-user canary.
  6. Soak it for hours and verify external behavior.
  7. Proceed one at a time with a host-memory ceiling and rollback point.

The proposed one-day estimate for diagnosing and repairing eleven real-user containers is too aggressive, especially if proper soak time is included.

Phase 1: parts should move ahead of broad Phase-0 rollout

The full architecture review need not block the first isolated diagnostic, but the following should precede repairing all real containers:

“A simple periodic restart-count check” is insufficient. Restart counts can reset on container recreation. Monitoring should key off stable instance identity and include event rate, readiness failure, host pressure, and absence of expected successful activity.

Fable’s “no leak found” claim is also narrower than it sounds. Disk usage and small logs do not establish absence of memory, connection, thread, file-descriptor, queue, or external-cost leaks.

Phase 2: /context and /context/writeback need safety design before implementation

For a family system handling children’s information, reminders, and mental-health-adjacent content, “reuse the existing /chat JWT check” is not enough.

Before implementation, specify and test:

The independent source check confirms ThreadingHTTPServer at api.py:6610, which makes concurrency isolation especially important. It confirms the structural precondition for the previously described race, but does not independently prove that the cross-user ledger bug has been fixed. Before adding two more stateful endpoints, inspect the actual current-user state handling and run concurrent two-user isolation tests.

“Dev engine only” is not automatically zero risk. Confirm that:

Phase 3: migration and hardening order needs adjustment

“Docker migration first” is ambiguous because it mentions dev, soak, and live profiles. Any migration of an existing gateway or live profile has nonzero risk.

Before the Hermes-heavy profile handles family data:

The minimum security and reliability subset of Phase 4 should move before Phase 3 is declared ready. Hardening cannot be wholly deferred until after the path is stood up.

Phase 4: too vague for a safety gate

“Ongoing” is not an acceptance criterion. Define explicit gates, such as:

Phase 5: testing depth is insufficient for this product

Longitudinal family simulation and Bharath’s dogfood instance are useful, but they do not cover the highest-consequence behaviors.

The plan needs tests for:

A real configured Bharath instance is not a substitute for these.

Phase 6: “low risk” is reasonable, but health is not proven by uptime alone

A month-running systemd service with no recent journal errors is process-stable, not necessarily end-to-end healthy. Before integration, verify:

Unsupported cost conclusion

Fable’s statement that the $15–40/day spend is “very likely concentrated almost entirely in chris-b” is not established. It has confirmed that one container aborts before useful work, but not that all broken containers always fail before model calls, nor that stable test instances generate no scheduled traffic. Empty usage tables make attribution weaker, not stronger.

Before making cost decisions, obtain provider-side usage keyed by API key/project/model where possible, or add per-instance attribution.

(d) Blocking objection before approval

Yes: do not approve the plan as an executable runbook in its current form

I would approve it only as a directional investigation plan, subject to mandatory amendments. The blockers are:

  1. No pre-change backup and restore gate.
    Real-user containers must not be recreated, restarted from an exited state, or experimentally modified until mounts, writable-layer state, DB records, and runtime configuration are preserved and one restoration is demonstrated.

  2. The OOM root cause is not established.
    V8 heap exhaustion is confirmed, but the workspace/skill-growth explanation is speculative. A ten-minute clean boot after increasing memory is not enough to rule out a leak or explicit heap setting.

  3. No aggregate memory-capacity plan.
    Raising eleven containers to 2–4 GiB on a 30 GB host could convert isolated V8 aborts into host-wide pressure affecting oimy-engine, Hermes, and other live services.

  4. “Risk to live traffic: none” is false for Phase 0 and not established for Phases 2–3.
    Real-user bots, shared host resources, external tokens, production data stores, and provisioning scripts are involved.

  5. The /context/writeback plan lacks tenant-isolation, idempotency, and concurrency requirements.
    This is particularly serious given the confirmed threaded HTTP server and the product’s sensitive family data.

  6. The seven “safe to stop” instances are not yet safe to delete.
    The plan must explicitly prohibit container removal, volume/workspace deletion, and DB-row deletion until archive retention and restoration criteria are met.

  7. Monitoring is too weak and too late.
    Alerting and capacity protection should be installed before broad real-user repair, not merely after it.

Minimum approval conditions

Bharath could approve execution once the plan is amended to require:

The central audit finding is real and important: the fleet is in severe operational distress, and pool-xoh3 is demonstrably dying from V8 heap exhaustion. But Fable moves too quickly from that valid finding to a uniform root-cause theory, a one-day fleet repair, and repeated claims of zero production risk. Those need correction before this becomes an approved production execution plan.

Call 2 — Independent Production-Readiness Assessment

Independent production-readiness assessment

Bottom line: I would not expand onboarding or move medication reminders onto this system yet. The current api.py product IP is worth preserving, but the system lacks reliable proof that each user turn is processed in the correct tenant/session and reaches the user. The six-week OpenClaw outage demonstrates that failures can persist indefinitely without detection.

1. What I would check before trusting it with real families

These are concrete release gates, not general recommendations.

A. Prove which code and infrastructure are actually serving every user

Build an inventory from the public entry points inward:

This is necessary because the system currently has at least two unrelated serving stacks, and the historical production source was not version-controlled. Until this inventory exists, a successful test may be testing a different path from the one a family uses.

For OpenClaw specifically, determine:

Stop the restart storm while preserving the relevant logs and container metadata for incident analysis.

B. Run an adversarial cross-tenant isolation test

The database retrospective does not prove that the threading.local() repair is complete. It only shows no known historical impact in the fields inspected.

Create two test families with unmistakable canary values, for example:

Then:

  1. Force concurrent requests through /chat using barriers inserted immediately before and after user/skill selection, constraint lookup, prompt construction, and writeback.
  2. Run thousands of deliberately interleaved A/B requests.
  3. Exercise exceptions and timeouts between each of those stages, because pooled threads are reused after a failed request.
  4. Inspect constructed prompts, retrieved anchors, entity-fact writes, deep-profile jobs, constraint-ledger writes, and responses—not just final text—for the other tenant’s canary.
  5. Repeat for two users in the same family and two separate families.
  6. Attempt to submit another user’s ID directly to /chat; verify that identity comes from authenticated gateway credentials, not an untrusted request field.

Also search the engine for every mutable singleton attribute, module-level cache, and background-job closure—not just _current_user_id and _current_skill_id. Any request identity, family identity, skill, topic, language, or conversation identifier held outside an explicit per-request object is another candidate leak.

A longer-term repair should replace implicit thread-local request state with an explicit immutable RequestContext/SessionContext passed into each subsystem. threading.local() will become unsafe again if the service later adopts async task concurrency, where multiple tasks can share one thread.

C. Prove the new session boundary is used everywhere

Construct a deterministic session-boundary matrix around the exact 30-minute rule:

For every case, capture and compare the session ID/window used by:

The pass condition is not merely “the reply looks reasonable.” Every subsystem must report the same authoritative session identifier and boundary.

Add permanent regressions for:

D. Test stale background work as a protocol, not just one bug

Every asynchronous result should carry at least:

Test these cases:

  1. Job completes after the user changes topics.
  2. Job completes after a new 30-minute session begins.
  3. Two jobs complete out of order.
  4. A job is retried and produces a duplicate result.
  5. The service restarts after computation but before delivery.
  6. Reconciliation/blending throws an exception.
  7. The originating user or conversation is deleted before completion.

The required behavior must be explicit: discard, store as a durable fact pending validation, or recompute against current context. It must never fall back to sending the raw stale result. “Could not reconcile” is not permission to deliver.

E. Establish end-to-end message accountability

Assign one immutable turn ID at Telegram ingress and track it through:

  1. Telegram update received.
  2. Update authenticated and deduplicated.
  3. Correct tenant/user resolved.
  4. /chat accepted it.
  5. Session/context snapshot selected.
  6. Model generation completed.
  7. Durable writes completed or failed.
  8. Telegram send attempted.
  9. Telegram returned success and a message ID, or a terminal failure was recorded.

Then test:

The system needs idempotency at ingress and a durable outbound outbox. A generated answer should not exist only in process memory. If delivery ultimately fails, operators must see the failed turn, and the product should give the user an honest failure/retry indication where possible.

The patched Hermes degraded-window behavior should be covered by this test even if Hermes is not adopted. Do not treat the direct site-package edit as deployed code: create a maintained fork or an automatically applied, hash-verified patch, and make CI fail if an upgrade removes it.

F. Verify medication reminders as a separate safety-critical workflow

The summary does not say how reminders are implemented, so this needs direct inspection before use. Test, at minimum:

No wording should imply that medication was taken merely because a reminder was sent. The reminder record needs an idempotency key and an audit trail of creator, edits, schedule, sends, failures, and acknowledgments.

G. Verify data access, deletion, and recovery

Run concrete authorization tests:

Then perform:

For child and custody-related data, the missing fact I would need is the exact authorization model: family-wide sharing versus private user scopes, guardian roles, and what happens after separation or custody-role revocation. That model must be defined before isolation can be meaningfully tested.

H. Load-test the actual stdlib server

Test the deployed ThreadingHTTPServer, not a mocked handler:

Measure active thread count, queueing latency, memory growth, open descriptors, and cross-user correctness under load. If a few slow requests can create unbounded threads or exhaust the DB pool, onboarding should remain capped until the HTTP boundary is replaced.

2. The single biggest reliability risk

The biggest risk is the absence of closed-loop, end-to-end operational accountability: the team cannot currently prove that a real user’s request was handled by the correct backend and successfully delivered, nor reliably detect when that stops happening.

The OpenClaw incident makes this a demonstrated failure, not a hypothetical concern:

This beats the other candidates because it makes every other defect effectively unbounded:

This is not merely “missing dashboards.” It is missing an operational truth source tying expected users and instances to health, inbound turns, processing, delivery, and durable records.

3. Should OiMy adopt the Hermes-heavy architecture?

No—not now. I would stabilize and simplify the existing api.py path rather than place Hermes in front of it.

Hermes does not address the failures that have actually occurred:

The proposed /context plus /context/writeback split would introduce a new consistency problem: Hermes would own turn/session state while api.py owns durable family context. Unless the protocol has snapshot versions, idempotent writes, conflict handling, and one explicit authority for session identity, the architecture recreates the exact “four definitions of current conversation” problem across a network boundary.

The simpler path I recommend

  1. Make api.py the sole supported serving path for now.
    Quarantine or retire the per-user OpenClaw fleet until there is a specific capability it uniquely provides and it passes the same production gates.

  2. Refactor request state explicitly.
    Create a typed RequestContext/SessionContext containing tenant, user, authoritative session ID, turn ID, history watermark, and skill. Pass it through the engine rather than relying on singleton fields or thread-local state.

  3. Separate transport from product logic without adding an agent framework.
    Put Telegram handling in a thin adapter that: - authenticates and deduplicates updates, - calls one stable engine interface, - records delivery through a durable outbox.

Add future iMessage or other channels as equivalent adapters. Multi-platform routing does not require a general-purpose agent framework.

  1. Replace ThreadingHTTPServer with a conventional production server after request state is explicit.
    A small FastAPI/Starlette or WSGI boundary can call the existing synchronous engine. Do not simply turn on async or multiple workers first: thread-local state and process-local caches must be eliminated or deliberately redesigned before changing the concurrency model.

  2. Add a durable job/outbox model.
    Async computation and outbound sends should be rows with turn/session IDs, context revisions, status, retry count, and terminal disposition. This directly addresses stale reach-back and silently dropped replies.

  3. Put all source, configuration migrations, and dependency patches under reproducible deployment control.
    Pin Hermes only where it is currently needed, and preserve the Telegram fix in a fork or a versioned patch with a CI assertion. Stop editing installed packages in place.

  4. Do not build /context as an HTTP boundary yet.
    First define and exercise it as an internal typed interface. If later scaling or organizational boundaries genuinely require a separate service, promote it to HTTP with: - authenticated tenant identity, - versioned schemas, - snapshot/context revision tokens, - idempotency keys, - conflict semantics, - strict field allowlists, - audit records.

Hermes can be reconsidered after the stable baseline exists and only if a measured prototype demonstrates a capability that thin channel adapters cannot provide. Its version should then be pinned and upgrades treated as migrations, not routine package bumps.

4. Monitoring and alerting gaps implied by the OpenClaw incident

Several cheap signals should independently have caught this within minutes or hours.

A. Container lifecycle alerts

Collect Docker runtime state and alert on:

A restart count in the tens of thousands means even a once-hourly reconciliation script would have found this.

B. Fleet reconciliation

The management plane should maintain an expected-instance list and compare it with runtime state every few minutes:

Alert if:

This must distinguish “container process exists” from “application can serve a request.” A crash-looping container may repeatedly appear as started.

C. External synthetic checks

From outside both VMs:

Run it every five minutes. At minimum, run one synthetic against each deployed image/configuration cohort; given only ~22 containers, checking every instance is inexpensive.

This would detect proxy routing, certificate, container, and application-startup failures that host-local health checks miss.

D. Message-funnel alerts

Count turns at each stage:

Alert on mismatches, for example:

This would have caught both the six-week fleet outage and the Hermes “generated but silently not sent” defect.

E. Logging-pipeline canaries

The empty message_logs, session_logs, and usage_tracking tables require their own checks.

After a scheduled synthetic conversation:

A table’s existence is not evidence of instrumentation. The synthetic must prove that running code writes to it.

F. Deployment health gates

After any redeploy:

  1. Observe each instance start successfully.
  2. Require readiness checks to pass.
  3. Send a synthetic turn through the public route.
  4. Verify its activity/cost rows.
  5. Verify outbound delivery.
  6. Compare fleet health and restart rates with the pre-deployment baseline.
  7. Automatically halt or roll back if the healthy count drops.

The mid-May deployment apparently had no such post-deploy verification.

G. Host-level resource and crash-loop signals

Alert on:

These are secondary signals, but tens of thousands of restarts should have been visible even without application instrumentation.

What the six-week miss tells us

The duration and scale imply more than one missing alert. They indicate that:

Therefore, other silent failures should be presumed possible now: dropped Telegram replies, expired credentials, failed profile extraction, non-running scheduled reminders, broken backups, stalled background jobs, and routes pointing to the wrong version. Each needs an active synthetic or reconciliation check; absence of user complaints is not a health signal, especially for newly signed-up families who may simply abandon the product.

Recommended immediate decision: freeze expansion, choose api.py as the single supported path, complete the tenant/session/delivery gates above, establish the end-to-end turn ledger and fleet reconciliation, and only then resume onboarding.