DOI: http://dx.doi.org/10.18203/2394-6040.ijcmph20194476

True-false analysis reveals inherent flaws in multiple true-false tests

Thomas Puthiaparampil, Henry Rantai Gudum, M. Mizanur Rahman, Rosalia Saimon, Isabel Fong Lim

Abstract


Background: Multiple true-false tests (MTF), a component of our assessment system, have consistently generated low scores and many failures. This was attributed to the negative marking scheme. However, no study was conducted to explore the issue further. Item analysis revealed that students omitted false options more frequently and answered them wrongly more frequently than true options. The aim of this study was to determine the performance discrepancy between true and false options of MTF tests and the reasons for such discrepancy and the poor performance of MTF in general.

Methods: The student performance of past 7 years of year-3 medicine end-of-posting examinations was analysed. The item analysis reports of 23 MTF tests were used to determine the significance of the differences in omission rates, correct-answer rates and the discrimination index of true and false options.

Results: There were statistically significant differences in the omission rates, correct-answer rates and discrimination index values of true and false options. This study revealed that the false options consistently let down student performance. Although negative marking could be partly blamed for the situation, no justification could be found for the use of false options to test knowledge.

Conclusions: Some publications endorse MTF, but many highlight its drawbacks. The use of false options in MTF was seen as an inherent defect in this instrument. As viable alternatives like VSAQ and Constructed Response Tests are in the horizon, we conclude that MTF ought to be discarded as an assessment instrument.


Keywords


Multiple true-false tests, True-false analysis, Flaws in MTF tests

Full Text:

PDF

References


Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. 2nd edition. Philadelphia: National Board of Medical Examiners; 1998: 22-25.

Jolly B. Assessment and examination. Adv Psychiatric Treat. 1999;5:405–14.

Schuwirth LWT, Van Der Vleuten CPM, Donkers HHLM. A closer look at cueing effects in multiple-choice questions, Med. Edu. 1996;30:44–9.

Case SM, Swanson DB. Constructing written test questions for the basic and clinical sciences. 3rd edition. Philadelphia: National Board of Medical Examiners; 2001.

Rasiah SMS, Isaiah R. Relationship between item difficulty and discrimination indices in true or false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singapore. 2006;35(2):67-71.

John A. For Multiple Choice Questions Medical Teacher. 1979;1:37-42.

John A. The MCQ Controversy: a Review Medical Teacher. 1981;3:150-6.

John A. Medical Teacher 25th anniversary series multiple choice questions revisited. Medical Teacher. 2004;26(2):110-3.

Leonard AB. Hints for students (and examiners) on answering MCQ questions of the multiple true-false type, Medical Teacher.1986;8(1):41-8.

Smart GA. The multiple choice examination paper. British J Hospital Med. 1976;15:131.

Tavakol M, Dennick R. Post-examination analysis of objective tests, Medical Teacher. 2011;33:447-58.

Premadasa IG. A reappraisal of the use of multiple choice questions. Medical teacher. 1993;15(2-3):237-42.

Hauer KE, Boscardin C, Brenner JM, van Schaik SM, Papp KK. Twelve tips for assessing medical knowledge with open-ended questions: Designing constructed response examinations in medical education. Medical teacher. 2019;5:1-6.

Sam AH, Hameed S, Harris J, Meeran K. Validity of very short answer versus single best answer questions for undergraduate assessment. BMC Med Edu. 2016;16(1):266.