Graduate Versus Undergraduate Interrater Reliability of the Landing Error Scoring System (LESS) and Less-Real Time

Abstract

The study sought to determine if the LESS and LESS-RT scoring criteria are reliable when scored by graduate and undergraduate kinesiology students with minimal experience. Eleven graduate (7 male: 28.29 ± 3.251 years; 4 female: 28.00 ± 4.082 years) and 19 undergraduate (7 male: 21.57 ± 1.512 years; 12 female: 21.42 ± 4.870 years) students participated as raters. Raters with minimal (< 2 hours) or no experience with the LESS watched 30 videos and evaluated jump-landing mechanics using the LESS and LESS-RT across four sessions (two per criteria). A 4-way repeated measures ANOVA analyzed interactions among trials, groups, videos, and scoring sheets. Intraclass correlation coefficient (ICC), coefficient of variation (CV), and minimal difference (MD) values were calculated, with an alpha level of 0.05. ICC values for the whole group and undergraduates (R = 0.102 – 0.780) demonstrated “poor” to “good” reliability, while graduate students (R = 0.356 – 0.814) demonstrated “poor” to “excellent” reliability. The CV for the whole group, graduate, and undergraduate students (14.24 – 29.90%), were all above the 10% threshold thus, reliable. Prior experience with the LESS may impact the quality of assessment, therefore, providing a single training session could drastically improve the quality of ratings, even for novice raters.