TY - JOUR
T1 - Identifying the response process validity of clinical vignette -type multiple choice questions
T2 - An eye-tracking study
AU - Specian Junior, Francisco Carlos
AU - Santos, Thiago Martins
AU - Sandars, John
AU - Amaral, Eliana Martorano
AU - Cecilio-Fernandes, Dario
N1 - Publisher Copyright:
© 2023 Informa UK Limited, trading as Taylor & Francis Group.
PY - 2023/2/25
Y1 - 2023/2/25
N2 - Introduction: Clinical vignette-type multiple choice questions (CV-MCQs) are widely used in assessment and identifying the response process validity (RPV) of questions with low and high integration of knowledge is essential. Answering CV-MCQs of different levels of knowledge application and integration can be understood from a cognitive workload perspective and this can be identified by using eye-tracking. The aim of the pilot study was to identify the cognitive workload and RPV of CV-MCQs of different levels of knowledge application and integration by the use eye-tracking. Methods: Fourteen fourth-year medical students answered a test with 40 CV-MCQs, which were equally divided into low-level and high-level complexity (knowledge application and integration). Cognitive workload was measured using screen-based eye tracking, with the number of fixations and revisitations for each area of interest. Results: We found a higher cognitive workload for high-level complexity (M = 121.74) compared with lower-level complexity questions (M = 51.94) and also for participants who answered questions incorrectly (M = 94.31) compared with correctly (M = 79.36). Conclusion: Eye-tracking has the potential to become a useful and practical approach for helping to identify the RPV of CV-MCQs. This approach can be used for improving the design and development of CV-MCQs, and to provide feedback to inform teaching and learning. Practice points Clinical vignette- type multiple choice questions (CV-MCQs) are widely used in assessment. Identifying the response process validity (RPV) of CV-MCQs with low and high complexity is essential. Low and high complexity CV-MCQs have different cognitive workloads that can be practically identified using eye-tracking. Eye-tracking has the potential for identifying the RPV of CV-MCQs.
AB - Introduction: Clinical vignette-type multiple choice questions (CV-MCQs) are widely used in assessment and identifying the response process validity (RPV) of questions with low and high integration of knowledge is essential. Answering CV-MCQs of different levels of knowledge application and integration can be understood from a cognitive workload perspective and this can be identified by using eye-tracking. The aim of the pilot study was to identify the cognitive workload and RPV of CV-MCQs of different levels of knowledge application and integration by the use eye-tracking. Methods: Fourteen fourth-year medical students answered a test with 40 CV-MCQs, which were equally divided into low-level and high-level complexity (knowledge application and integration). Cognitive workload was measured using screen-based eye tracking, with the number of fixations and revisitations for each area of interest. Results: We found a higher cognitive workload for high-level complexity (M = 121.74) compared with lower-level complexity questions (M = 51.94) and also for participants who answered questions incorrectly (M = 94.31) compared with correctly (M = 79.36). Conclusion: Eye-tracking has the potential to become a useful and practical approach for helping to identify the RPV of CV-MCQs. This approach can be used for improving the design and development of CV-MCQs, and to provide feedback to inform teaching and learning. Practice points Clinical vignette- type multiple choice questions (CV-MCQs) are widely used in assessment. Identifying the response process validity (RPV) of CV-MCQs with low and high complexity is essential. Low and high complexity CV-MCQs have different cognitive workloads that can be practically identified using eye-tracking. Eye-tracking has the potential for identifying the RPV of CV-MCQs.
UR - http://www.scopus.com/inward/record.url?scp=85149344011&partnerID=8YFLogxK
U2 - 10.1080/0142159X.2023.2182662
DO - 10.1080/0142159X.2023.2182662
M3 - Article
C2 - 36840707
AN - SCOPUS:85149344011
SN - 0142-159X
VL - 45
SP - 845
EP - 851
JO - Medical Teacher
JF - Medical Teacher
IS - 8
ER -