Comparing Dichotomous and Polytomous Instruments in Cdm-Based Swimming Technique Assessment

Abstract

Assessing complex motor skills across physical performance domains poses distinct methodological challenges for applying the Cognitive Diagnostic Model (CDM), particularly regarding the response format of observational instruments. This study compared inter-rater reliability, model fit, diagnostic specificity, and criterion validity between dichotomous and polytomous instruments for assessing front crawl technique. A total of 280 physical education students from Universitas Negeri Surabaya participated. The dichotomous instrument (10 items, three attributes: Arm Action, Body & Leg, Breath & Synchronization) was adapted from the Crawl Technique Observation Sheet and analyzed using GDINA. The polytomous instrument (20 items, four dimensions) was analyzed using pGDINA with a three-attribute Q-matrix, after Body Position and Leg Action were mapped into a single Body & Leg attribute based on biomechanical justification, structural equivalence, and sample adequacy. The dichotomous instrument showed higher inter-rater reliability (Fleiss’ κ = 0.663) than the polytomous instrument (ICC = 0.538), with a Classification Accuracy of 0.924 and higher validity against swimming speed (ρ = 0.507 vs. 0.288). The polytomous instrument produced higher diagnostic entropy (2.362 vs. 1.568 bits) and a more even latent profile distribution (largest profile 40.7% vs. 62.5%), indicating better diagnostic specificity. Cross-instrument profile consistency was low (Cohen’s κ = 0.370), with 31.4% of borderline students classified differently by both instruments. A trade-off emerged: the dichotomous instrument excelled in reliability, classification accuracy, and criterion validity, whereas the polytomous instrument excelled in diagnostic depth, confirming that CDM applications in motor skill assessment face challenges distinct from cognitive education contexts.