Abstract

Research on multiword expressions (MWEs)—including collocations, idioms, and lexical bundles—has advanced substantially; however, most assessments focus on accuracy- and knowledge-oriented measures rather than the awareness processes underlying that knowledge. This article reports the development and initial validity evidence of the Collocation Awareness Assessment Tool (COLLAT), a theoretically grounded instrument operationalizing collocation awareness across three dimensions: cognitive noticing, metacognitive awareness, and metalinguistic awareness through identification, self-report, and binary forced-choice judgment tasks, respectively, with the first two accompanied by written justifications. Data from 162 Vietnamese university EFL learners were analyzed within an argument-based validation framework. Signal detection analysis operationalized cognitive noticing as the discrimination of target collocations from non-target word combinations in context, distinguishing perceptual sensitivity (d′) from response criterion (c). The self-report component exhibited an interpretable unidimensional structure with acceptable reliability. Rasch modeling of the judgment task indicated orderly item difficulty, although targeting favored lower-to-mid ability and precision was limited at the upper range. Mixed-effects modeling supported the cross-model consistency of item difficulty estimates. Analyses of relationships among the three COLLAT components revealed limited alignment between quantitative performance indices and qualitative reasoning depth. Overall, findings provide initial support for interpreting cognitive noticing, metacognitive awareness, and metalinguistic awareness as related but potentially separable aspects of collocation awareness. COLLAT may therefore complement knowledge-focused collocation assessments by providing diagnostic and research-oriented evidence about how learners notice, evaluate, and explain collocations.