Boojee Companion Care
Regulatory & Legal Analysis

The 21st Century Cures Act (2016) added FD&C Act §520(o), carving certain software functions out of the "device" definition. Two carve-outs apply here, and we design to satisfy both:

• §520(o)(1)(B) — general-wellness software: intended for maintaining/encouraging a healthy lifestyle and unrelated to the diagnosis, cure, mitigation, prevention, or treatment of a disease. Basis for framing loneliness/mood check-ins as wellness.
• §520(o)(1)(E) — Clinical Decision Support (CDS): non-device if it meets all four criteria (below). This is our backstop for the depression/anxiety instruments, which name recognized conditions.

FDA re-issued both interpreting guidances in early 2026, superseding the 2019 (General Wellness) and 2022 (CDS) versions. Commentary reads the 2026 revisions as broadening the non-device space, while stressing that "black-box" algorithmic output remains a device risk and that transparency of inputs/logic is expected — which favors our design.

The four CDS non-device criteria (§520(o)(1)(E)) — all required

• 1. Not intended to acquire/process/analyze a medical image, IVD signal, or a pattern/signal from a signal-acquisition system. ✔ We take self-reported questionnaire answers only.
• 2. Intended to display/analyze medical information about a patient (or reference guidelines). ✔ We display the instrument's own published band.
• 3. Intended to support/provide recommendations to a health care professional on prevention/diagnosis/treatment. ✔ Output routes to the licensed reviewer.
• 4. Intended to let the HCP independently review the basis so they do not rely primarily on the software. ✔ We show items, raw answers, published algorithm, and citation.

The PHQ-9 item-9 / crisis feature — surface, don't assess

Automated suicide-risk prediction is exactly the black-box clinical decision that reads as a device and forfeits criterion 4. So we do not compute a risk score. PHQ-9 item 9 > 0 (or self-harm language) is a trigger to surface 988 + connect a human — a safety net, explicitly not a risk assessment. The structured assessment (C-SSRS / Columbia Protocol) stays with the human clinician.

Features/claims that would tip us into device territory

• Diagnostic language — labeling the person with a condition or code.
• Treatment recommendations — therapy, medication, dose, "start therapy for X."
• A proprietary/black-box risk score — especially a homegrown suicide-risk model (forfeits criterion 4).
• Autonomous / time-critical clinical decisions the user acts on without a human ("you're safe, no action needed").
• Image / IVD / physiologic or affect-signal analysis (voice-stress, facial affect) — forfeits criterion 1, reads as SaMD.
• Efficacy or clearance claims — "FDA-cleared," "clinically proven to reduce depression," "treats loneliness." Claims set intended use.

Every state licenses the practice of medicine and psychology and prohibits doing either unlicensed (in New York, unlicensed practice of psychology is a felony). The definitions turn on diagnosing and treating. Our product screens, monitors, engages, and routes to licensed humans — the wellness / peer-support side of the line. It never diagnoses or generates a treatment plan.

The 2025–2026 AI-therapy / AI-companion statutes

• Illinois — WOPR Act (HB 1806), Aug. 2025: bans AI-delivered therapy unless a licensed professional is in charge; bars AI diagnoses/treatment plans; fines to $10,000. Exempts peer support, self-help, and educational resources — the exemption we map to.
• Nevada — AB 406 (Jul. 2025): bars AI from standing in for a counselor/psychologist; fines to $15,000.
• Utah — HB 452 (Mar. 2025): permits with AI-disclosure, no data sale, marketing limits, and a safe harbor — a national compliance floor to adopt.
• New York — AI Companion Models law (eff. Nov. 5, 2025): requires clear/periodic "you're talking to AI" disclosure and crisis-detection → 988 referral; AG enforcement, up to $15,000/day; no private right of action.
• California — SB 243 (core eff. Jan. 1, 2026): similar companion-safety duties plus a private right of action — materially higher litigation risk.

Structural protections: a named licensed clinical director owning clinical judgment (positive screens route to that human); and, before contracting clinicians, a counsel-designed friendly-PC / MSO structure to address the corporate-practice-of-medicine doctrine.

Tarasoff duty to warn / protect

Classic Tarasoff attaches to a licensed professional–patient relationship and a serious danger to an identifiable third party. Most AI self-harm content is diffuse with no identifiable third-party victim, so the classic trigger often won't fire on the AI itself — but the duty can attach to our clinician the moment an escalation reaches them. The SOP must separate (a) self-harm (surface 988 + notify consented caregiver + queue a human alert) from (b) credible threat to an identifiable third party (may trigger the clinician's state duty-to-protect).

Mandatory elder-abuse reporting — a senior-care product's special exposure

Elder-abuse reporting laws sweep broadly: caretakers and persons in a position of trust are commonly mandated reporters, and self-neglect is a reportable category in most states, filed to Adult Protective Services, usually immediately. Whether Boojee's clinical/care staff become mandated reporters in a given state is an open question counsel must resolve — and the SOP must include an elder-abuse / self-neglect reporting pathway with per-state APS routing.

Suicide-risk standard of care

The high-consequence failure is the response: missed crisis, delayed escalation, or false reassurance. Garcia v. Character Technologies lets a wrongful-death suit proceed on product-liability (failure-to-warn / design-defect) and deceptive-practices theories — including that the bot implied it was a real person / licensed professional. Defensive design: always escalate, never false-reassure, fail loud, log everything, never let the AI make the safety call.

The escalation SOP must contain

• Enumerated triggers with separate self-harm / third-party-threat / elder-abuse branches.
• Immediate automated response (surface 988 call/text; consented caregiver notice; never minimize).
• Named responsible humans, backup/on-call, and explicit after-hours coverage for a 24/7 population.
• A stated response-time SLA with escalation if unmet.
• A duty-to-warn branch and an APS-reporting branch.
• Fail-safe behavior if escalation infra is down; append-only documentation; up-front informed consent; periodic drills/audits.

Liability & insurance (high level)

Exposures: missed crisis, false reassurance, over-reliance by isolated seniors (the exact concern behind the FTC 6(b) companion inquiry and the NY/CA laws), data breach, and misrepresentation. Mitigate with escalate-always design, repeated "not a substitute for people/professional care" disclosure, informed consent, and claims discipline. Limitation-of-liability clauses are frequently unenforceable against wrongful-death / gross-negligence / consumer-protection claims — they reduce but do not eliminate exposure. Carry tech E&O, professional liability (E&O), general liability, and cyber — and confirm policies do not exclude AI-driven harm, suicide/mental-health claims, or bodily-injury-from-software.

Colorado AI Act (SB 24-205 → SB 26-189)

Duties on deployers of high-risk AI making consequential healthcare decisions: reasonable care against algorithmic discrimination, impact assessments, disclosures, human oversight. Original Feb. 1, 2026 date was postponed; the repeal-and-reenact (SB 26-189) is effective Jan. 1, 2027. A carve-out exists for HIPAA-covered entities whose AI recommendations require a provider to act. Counsel to confirm our deployer status; our human-in-the-loop design aligns with the carve-out logic.

EU AI Act — only if we serve the EU

Health-AI is high-risk where the AI is or is a safety component of an MDR/IVDR-classified medical device — the AI-Act tier is tied to the device class. Because we design to stay a non-device, the parallel EU argument is that we are not high-risk on that ground — but this must be re-run under EU MDR (its device definition differs). Article 50 still requires AI-interaction disclosure, and adding emotion recognition would attach its own transparency duty — another reason not to add affect inference. (High-risk deadlines are being pushed toward Dec. 2027 / Aug. 2028, subject to pending legislation.)

FTC — deceptive AI claims + Health Breach Notification Rule

The FTC's "Operation AI Comply" targets unsubstantiated AI claims (it settled with Pieces Technologies over overstated clinical-AI accuracy). Any diagnostic-accuracy / "AI clinician" / efficacy claim must be substantiated or not made — our honesty posture (instruments validated, product not; no clearance) is the FTC-safe stance. Separately, the 2024 Health Breach Notification Rule amendments (eff. Jul. 29, 2024) extend it to many non-HIPAA health apps.

Telehealth / scope

The AI (screen / monitor / connect) should sit outside telehealth regulation. The human escalation layer — when a licensed clinician engages an enrollee remotely — is telehealth and pulls in licensure-by-enrollee-state, telehealth consent, and standard-of-care for that clinician. Build that layer to telehealth norms.

Design and claims constraints we hold ourselves to. This is a self-imposed posture, not an FDA determination or a compliance certification.

• No diagnosis, no treatment plans to the user — ever. Never label a condition; never recommend therapy/meds/dose.
• No proprietary/black-box risk score — especially no homegrown suicide-risk model. Only published instruments' own bands, with citation.
• Full transparency for the human reviewer — items, raw answers, published algorithm, source citation (preserves CDS criterion 4).
• Self-report only — no images, IVD signals, or physiologic/affect-signal analysis (preserves CDS criterion 1).
• Human-in-the-loop by design — positive screens route to a licensed human; the AI never closes the clinical loop or makes a time-critical call.
• Crisis feature = surface 988 + a human, explicitly labeled "not a suicide-risk assessment"; escalate always; never false-reassure; fail loud; log everything.
• Clear, periodic "I'm AI, not a human, not a licensed clinician" disclosure (UT / NY / CA; repeated for seniors).
• Stay inside the peer-support / self-help / educational exemption (IL WOPR) in behavior and marketing.
• Claims discipline — market as screen / monitor / connect; never diagnose / treat / cure / FDA-cleared / clinically proven.
• No sale of intimate mental-health data; strong privacy + breach posture (UT; FTC HBNR / HIPAA).
• Appoint a named licensed clinical director and a written escalation SOP before any real enrollment.
• Maintain a 50-state operating map and re-check it as new AI-therapy bills pass.
• Confirm insurance (tech E&O, professional E&O, cyber) with no AI/suicide/software-bodily-injury exclusions.

These are open items and unresolved risks. Stating them is load-bearing; they are not resolved by this dossier.

• No licensed clinical director is appointed yet — increasingly the condition of legality under the new laws, not merely best practice. Our #1 disclosed gap.
• No written escalation SOP exists yet — the highest-priority defensibility gap; must be authored/owned by the clinical director.
• Mandated-reporter status is undetermined per state for elder abuse / neglect / self-neglect; no APS reporting pathway is documented.
• After-hours / on-call crisis coverage is undefined — a 24/7 population with a business-hours human queue is a serious gap.
• No E&O / professional-liability / cyber insurance confirmed in force.
• HIPAA BAA not yet executed — governing breach regime (FTC HBNR vs. HITECH) is unsettled until it is.
• CDS carve-out for a consumer/senior-facing tool is untested by us — the guidance is written around clinician-facing software; counsel must confirm the theory holds where the "HCP" is a downstream care manager.
• The 2026 FDA guidance text should be verified against primary source (not only secondary summaries) before the intended-use statement is finalized. No FDA pre-submission / 513(g) is on file.
• No maintained 50-state AI-therapy operating matrix; CPOM contracting structure unresolved.
• Conversational-content governance is needed so the companion doesn't drift into therapy-like advice in practice (which would forfeit the exemptions).
• Crisis-language detection accuracy (false-negatives = missed crises) is itself safety-critical and needs validation + monitoring.
• EU exposure is dormant (U.S.-targeted); if we expand, a fresh EU-MDR + GDPR analysis is required — the U.S. wellness conclusion does not import.