How survey method can change what cannabis use data reveal

Reliable measurement of adolescent substance use is critical for accurately characterizing developmental risk, yet the consistency of commonly use self-report methods remains uncertain. Here, consistency of past 30-day substance use self-report data (alcohol, cannabis, nicotine) was compared between the timeline follow-back and monthly retrospective surveys in a sample of adolescents and young adults. N=94 participants (19-21-years-old, 62% female) were drawn from an ongoing longitudinal study. Measurements consisted of in-person timeline follow-back interview and mobile monthly retrospective surveys assessing past 30-day substance use; these timelines were harmonized to assess full overlap in reporting between modalities. Dimensions of reliability between modalities and concurrent validity with measures of substance use consequences, craving, and use disorder symptoms were analyzed. Across substances (alcohol use days, cannabis use episodes, nicotine use episodes) rank correlations ranged from strong to very strong; however, significant within-person differences were observed for nicotine and cannabis. Nicotine use had poor agreement between the in-person and mobile survey modalities (ICC=0.30), while alcohol (ICC=0.91) and cannabis (ICC=0.88) were excellent and good, respectively. Concurrent validity differed between modalities for cannabis symptoms and problem outcomes but were comparable for alcohol and nicotine. Findings reflect nuanced measurement considerations for substance-specific patterns. While largely no differences between modalities and within-subjects were observed for alcohol use days, self-report differences emerged for cannabis and nicotine use, depending on the outcome assessed. Implications are discussed in the context of prospective, longitudinal studies of adolescents and young adults, while attempting to reduce participant burden and maintain reporting accuracy.