Ethan Marshall

Money Can't Buy Agreement: Inspector for LLM Counseling Monitors

Ethan Marshall, Ubbo Visser, Christine Lisetti

Workshop Paper In Progress

An AI counselor in a live session needs a monitor that catches broken rules as they happen, and the usual monitor, a large language model acting as a judge, is slow and expensive. This paper asks which quick, inexpensive checks can do that job as well as an expert would. It puts a range of labelers, from small local models to frontier LLMs, against motivational-interviewing conversations coded by experts, and trusts each one, label by label, only where a confidence bound puts its error rate at an expert's level. An inspector interface replays sessions from the AnnoMI dataset through the eEVA virtual counselor and shows what each choice would mean for speed and cost.

• Each monitor is certified label by label, only where its error rate is at an expert's level
• A mix of quick local models and stronger LLMs instead of one frontier model for everything
• An inspector interface for seeing the speed and cost of each choice in the eEVA virtual counselor
• Workshop paper in progress for HUMANE-LLM at ACM IUI 2027 (Helsinki, February 2027), available on request

Conversational AI safety

My Ph.D. research at the University of Miami, advised by Dr. Ubbo Visser.

Conference Paper Under Blind Review

I work on what happens when a language model is deployed somewhere the stakes are human, like a counseling or companion chatbot. That means instruments, built with clinicians, that measure whether a model is actually following the protocols a good counselor would, and lightweight monitors that catch problems during a conversation instead of in an audit afterward. I'm advised by Dr. Ubbo Visser in the Department of Computer Science.

• Clinician-validated fidelity instruments adapted for machine-generated text
• Lightweight runtime monitors that flag protocol violations during a conversation, not in an audit afterward
• First-author paper under review (2026), available on request

Chorale

Playing tests for music teachers: assign once, sync everywhere, including Google Classroom.

App Store v1.0.0 Web gochorale.com

Chorale lets music teachers assign playing tests, collect recordings, and export grades in one place. The iOS app goes beyond the browser with on-device recording and App Clips that open an activity in seconds, and everything stays in sync with Google Classroom.

• An iOS app and a web app at gochorale.com
• Deep sync with the Google Classroom API
• App Clips launch an activity without installing anything

FBFD

A field reference app built for the Franklin-Bingham Fire Department in Michigan.

App Store v1.1.0 Google Play v1.1.0

A custom React Native app for the Franklin-Bingham Fire Department in Michigan, built front end to back end, that gives firefighters one-tap access to the phone numbers, locations, and data they need on a call.

• 300+ critical data items with quick actions for calls and directions
• Usability-tested with firefighters, deployed through iOS and Android enterprise distribution
• Admin roles for managing shared content and access

CORE

A clinical-research access platform. First place at the Michigan Ross Business+Tech Innovation Jam and a U-M Center for Entrepreneurship Fuel Award.

Ross Innovation Jam 1st Place LinkedIn Read More

CORE connects people to clinical research. Researchers publish studies on the web or in the apps, and participants find studies that fit them through a matchmaking system. I co-founded it and led the engineering.

• 1st place, Michigan Ross Business+Tech Innovation Jam ($7,000)
• U-M Center for Entrepreneurship Fuel Award ($7,500)
• SwiftUI apps and a Next.js platform