Portability & Lessons
Harbor Ridge isn't Portable. The Process that Built it is.
The benchmark result is on Methodology & Evaluation in full. This page is about what I took from it — what doesn't transfer out of a synthetic testbed, what does, and what I'd do differently for a real organization.
What Doesn't Transfer
Harbor Ridge itself is not portable to another healthcare vertical. Its schema, synthetic facility, and benchmark results belong to this test environment.
What Does Transfer
What is portable is the process I demonstrated by building it:
For another organization, I would rebuild that process around its actual operating reality rather than transplant Harbor Ridge and pretend healthcare organizations are interchangeable.
The Analytical Limitations
The V0.2 blind evaluation missed both planted failures while the Healthy Baseline remained clean. That clean result is useful evidence about the control test — it does not establish general reliability.
A later diagnostic suggested experimental signal-to-noise structure may have contributed, particularly because one affected cohort ultimately contained only roughly three to eight admissions per month. That explanation is directional, not proven.
I deferred another evaluation iteration rather than repeatedly modify the experiment in pursuit of a passing result.
So I would not present V1 as a production-validated, generally reliable autonomous diagnostic system.
What I Learned
-
Useful discovery and successful diagnosis are not the same claim.
The Evidence Engine surfaced a genuine OON collections-realization disparity. It did not reliably recover the planted root causes. I built the evaluation to keep those two facts separate rather than let one stand in for the other.
-
A clean control result is real evidence, but it isn't proof of reliability.
The Healthy Baseline stayed clean when no planted failure was present. That's useful — it means the system didn't invent a problem where none existed. It doesn't mean the system would stay right under different conditions.
-
When an evaluation produces a result I didn't want, the discipline is to preserve it before investigating why.
I froze Ground Truth and the evaluation protocol before running the blind sessions specifically so I couldn't redefine failure after seeing the outcome. When the misses came in, the diagnostic came after the freeze, not instead of it.
-
Portability comes from rebuilding around another organization's reality, not transplanting a testbed.
Harbor Ridge's schema and scenarios are specific to this synthetic environment. What transfers is the discipline of mapping a real organization's own systems before asking an AI to reason about them.
What This Adds Up To
What I can demonstrate is the complete build-and-evaluate process — including what I did when validation exposed defects, when AI surfaced something I hadn't planted, and when the final benchmark failed.