# Measurement Plan — Honest Outcome Measurement for Boojee Companion Care

**Prepared for:** Clinical Director, IRB, payer clinical reviewers
**Purpose:** define how we will measure outcomes *honestly* — what each endpoint is, what change counts
as meaningful, what confounds threaten inference, and **exactly what claim each study design can and
cannot support.** This plan is deliberately conservative: it is designed to *not* overstate.

---

## 1. Endpoints

### Primary
- **UCLA-3 change** (loneliness) — the target construct. Range 3–9.
- **PHQ-9 change** (depression severity) — where a PHQ-2 pre-screen is positive (≥3). Range 0–27.

### Secondary
- **GAD-7 change** (anxiety) — directional secondary; weaker change-anchor evidence (treat as signal).
- **Engagement** — sessions/week, session length, retention/attrition, feature use. Engagement is a
  **process** measure, **not** a clinical outcome; high engagement ≠ benefit and must never be presented
  as an efficacy result.

### Safety endpoints (always monitored, never a "benefit")
- PHQ-9 **item 9 > 0** events and time-to-human-contact.
- Crisis-language detections and escalation completion.
- Adverse events / worsening (score deterioration beyond reliable-change threshold in the *worse*
  direction).

---

## 2. Meaningful-change thresholds (per instrument)

We use **published individual-change anchors** where they exist and **refuse to invent them** where they
do not.

- **PHQ-9:**
  - **MID ≈ 5 points** (minimal important difference; ~2 SEM).
  - **Reliable change (Jacobson–Truax) ≈ ≥6-point** reduction = change unlikely to be measurement noise.
  - **Clinically meaningful response** commonly defined as ≥50% reduction from baseline; **remission** as
    PHQ-9 < 5.
  - We will report **both** MID-based and reliable-change-based responder rates, per person, not just group
    means.
- **UCLA-3:**
  - **No established MID or reliable-change value.** With only 7 possible total scores and α ≈ 0.72, the
    scale has limited change resolution. We will **not** assert a UCLA-3 MID. We will report the **full
    distribution of change** and, if we compute an RCI, we will derive SEM from **our own sample's**
    reliability and present it as *exploratory*, clearly labeled.
- **GAD-7:**
  - No clean MID/reliable-change equivalent to PHQ-9. Report as **directional** with distribution of change;
    do not claim calibrated clinical change.

**Reliable Change Index (Jacobson–Truax) method (for reference):**
RCI = (x_post − x_pre) / S_diff, where S_diff = √(2 · SEM²) and SEM = SD_pre · √(1 − reliability).
|RCI| > 1.96 → change unlikely due to unreliability (p < .05). We compute this **per participant**, using
instrument-specific reliability, and report **counts of reliably improved / no-change / reliably worsened.**

---

## 3. Confounds that threaten any "it works" inference

1. **Regression to the mean (RTM):** users enroll when distressed; re-test scores drift toward the mean
   **with or without any intervention.** This is the single biggest threat to a naive pre/post claim.
2. **Natural history / spontaneous remission:** loneliness and depression fluctuate; life events (a visit,
   a season, a health change) move scores.
3. **Response / demand bias & social desirability:** users tell a warm companion they feel better. Directly
   inflates self-report — the flaw that makes ElliQ-type numbers untrustworthy (see `evidence-base.md`).
4. **Practice / familiarity effects** from repeated identical items (see instrument dossier §4).
5. **Selection & attrition bias:** the users who stay engaged and keep completing measures are
   systematically different (healthier, more motivated). Per-protocol analyses overstate benefit;
   **intention-to-treat + attrition reporting** are mandatory.
6. **Placebo / attention effects:** *any* attention improves self-report; without a comparator we cannot
   separate "the companion" from "someone is paying attention."
7. **Mode-of-administration effect:** conversational delivery itself may shift scores (instrument dossier §5).
8. **Co-intervention / confounding by care:** users may start therapy, medication, or gain social contact
   concurrently.

---

## 4. Why observational / pre-post ≠ causal (stated plainly)

A single-arm pre/post improvement is **fully explained by RTM + natural history + demand bias + attention**
without the product doing anything. **Correlation of use with improvement is not causation.** We will not
present uncontrolled pre/post deltas as evidence that the companion *caused* improvement. Uncontrolled data
is legitimate for **feasibility, engagement, safety-signal, and hypothesis-generation** — nothing more.

---

## 5. Design ladder — what each design can HONESTLY claim

| Design | Can honestly support | Cannot support |
|---|---|---|
| **Uncontrolled pre/post (single arm)** | Feasibility, acceptability, engagement, safety signals, hypothesis generation | Any causal efficacy claim (confounded by RTM/natural history/demand) |
| **Concordance / fidelity study** (conversational vs standardized administration, within-subject, counterbalanced) | Whether our measurement mode is equivalent (ICC, bias, Bland–Altman limits of agreement) | Efficacy — it validates the *ruler*, not the *treatment* |
| **Pre/post with a waitlist or attention control (quasi-experimental)** | A *provisional* effect estimate, partly controlling attention; still vulnerable to selection | Strong causal claim |
| **RCT (randomized, ideally vs active/attention control, blinded outcome assessment where feasible)** | **Efficacy** on UCLA-3 / PHQ-9 change | Long-term durability beyond follow-up; generalization beyond the trial population |
| **RCT + adequately powered + preregistered + ITT** | A payer-grade efficacy claim with effect size and CI | — |

**Our sequence:** (1) fidelity/concordance study of conversational administration → (2) IRB-reviewed
feasibility single-arm (engagement + safety + provisional signal, explicitly labeled non-causal) →
(3) controlled trial (waitlist/attention → RCT) before **any** efficacy claim reaches marketing, payers,
or the public.

---

## 6. Analysis & reporting commitments

- **Preregister** primary endpoints, thresholds, and analysis before any efficacy study.
- **Intention-to-treat** primary analysis; report attrition transparently; per-protocol only as sensitivity.
- **Per-participant responder analysis** (MID + reliable change), not just group means — group means hide
  who actually changed.
- Report **effect sizes with confidence intervals**, not p-values alone; pre-specify a minimal effect
  worth claiming.
- Report **all** endpoints (no cherry-picking), including null and adverse results.
- **Blind outcome assessment** where feasible; use standardized self-administration for the *outcome*
  measurement even if the product delivers conversationally, to avoid mode confound in the endpoint.
- **Independent / clinical-director sign-off** on any outcome statement that leaves the building.

---

## 7. Claim boundary (must match `evidence-base.md`)

Until a controlled trial exists, the only defensible claims are about the **problem** (loneliness is a
serious health risk), the **instruments** (validated screeners), and the **process** (we screen, monitor,
and route to humans). **Efficacy of the companion itself remains unproven and will be described that way**
everywhere.

---

## Sources
- PHQ-9 monitoring / sensitivity to change (Löwe 2004) — https://pubmed.ncbi.nlm.nih.gov/15550799/
- MID and Reliable Change Index for PHQ-9 — https://novopsych.com/support/user-guide/reliable-change-index-rci/
- Jacobson–Truax reliable change methodology — https://pubmed.ncbi.nlm.nih.gov/15637779/
- Defining successful PHQ-9 outcome (methods comparison) — https://www.sciencedirect.com/science/article/abs/pii/S0165032710003666
- ICT-for-loneliness null meta-analysis (design cautionary basis) — https://pmc.ncbi.nlm.nih.gov/articles/PMC8692663/