Analysing the psychometric properties of the Botswana junior certificate examination agriculture multiple-choice items using item response theory

Abstract

The main purpose of this study was to analyse the psychometric properties of the Junior Certificate Examination (JCE) agriculture multiple-choice items using item response theory. In addition, the study examined Differential Item Functioning (DIF) in relation to sex, location, and school type. The study adopted the descriptive research design. Utilising the census approach, 123,218 candidates‟ responses with 4,928,720 cases of data points from the 2018-2020 examination period was used for the study. Secondary data on candidates‟ responses to each item were obtained from Botswana Examination Council (BEC). The 3-parameter logistic model of IRT was utilised for purposes of data analysis. Raju‟s procedure was employed for the DIF determination using DifR package within the R-studio environment. The study revealed that items had varied levels of difficulty, 15% to 20% of the items exhibited poor discrimination, and some items had high guessing indices. It was also revealed that agriculture science items had both uniform and nonuniform DIF across demographic variables. The study concluded that the contribution of items to student achievement in agriculture science was good with few items having validity and reliability concerns. It further concluded there exists DIF across all three demographic variables of gender, location, and school type. Resource disparities and sex-based cultural practices contribute to most DIF items. It was recommended that BEC should consider validating their examination items using IR.