AI counseling shows promise for substance users, including cannabis

Large language models (LLMs) have the potential to expand access to Motivational Interviewing (MI), but little is known about whether AI-delivered counseling reproduces conversational processes associated with motivation for behavior change. Change talk (CT), sustain talk (ST), and specific counselor behaviors are central components of MI theory, yet their occurrence and associations within AI-delivered counseling remain understudied. We conducted a secondary analysis of transcripts from a prospective pilot study of an AI Motivational Interviewing Counseling Agent (MICA) delivered to Emergency Department patients with risky alcohol, cannabis, or nicotine use. Participants completed a text-based counseling session with MICA (GPT-4). MICA utterances were coded using the Motivational Interviewing Treatment Integrity (MITI) framework and participant utterances were coded using the Motivational Interviewing Skill Code (MISC). We examined differences in counselor and participant language across substances, participant- and counselor-level predictors of change talk balance (CT/[CT+ST]), and exploratory associations between CT balance and post-session changes in readiness to change. Ninety-eight participants completed 105 counseling sessions. MICA demonstrated a consistent MI-consistent conversational style characterized by frequent use of open questions, reflections, and affirmations, with no identified MI-inconsistent behaviors. When expressed as proportions of total utterances, MICA behaviors did not differ significantly across alcohol-, cannabis-, and nicotine-focused sessions. Participant motivational language varied by substance, with nicotine-focused sessions showing higher CT balance and cannabis-focused sessions showing greater ST. In multivariable analyses, higher CT balance was independently associated with a greater proportion of complex reflections (β = 1.04, p < 0.001), a greater proportion of open questions (β = 0.67, p = 0.018), the presence of collaboration-focused statements (β = 0.12, p = 0.013), greater session length (β = 0.004, p = 0.010), and higher baseline readiness to change (β = 0.045, p < 0.001). In exploratory analyses of participants with both pre- and post-session readiness assessments, higher CT balance was associated with greater improvement in readiness to change following counseling (β = 1.02, 95% CI 0.33-1.71, p = 0.004). An AI counseling agent generated conversational processes broadly consistent with MI theory. Reflective depth, open questioning, and collaborative language were associated with greater participant change talk, and higher change talk balance was associated with greater improvement in readiness to change. These findings support the use of process-oriented frameworks to evaluate and optimize AI-delivered behavioral interventions.