A Randomization Test for the Detection of Differential Item Functioning

Abstract

A test question is said to contain Differential Item Functioning (DIF) if people who have the same ability level but are from different subpopulations have unequal probabilities of answering correctly. Detection of DIF is crucial to maintaining test fairness. Numerous DIF detection methods exist; none are ideal across a wide range of settings. I introduce a randomization test for DIF detection, and compare its performance to that of the Mantel-Haenszel, Logistic Regression and Lasso-based methods under a Rasch model using Monte Carlo simulations.