Domain-specificity of research competencies in the social sciences: Evidence from differential item functioning

Abstract

To investigate the domain-specifi city of research competencies, higher education students from the social sciences were assessed with a standardized test in four disciplines: (a) sociology, (b) political science, (c) educational studies, and (d) psychology. The measure covered declarative and procedural knowledge of research methods, methodology, and procedures. Quantitative and qualitative research traditions were represented equally by test items. The domain-specifi city of the measure was examined by detecting and explaining diff erential item functioning (DIF) between the disciplines. It was hypothesized that due to diff erences in opportunities to learn (OTL), students from diff erent disciplines responded diff erently to subgroups of items. As expected based on the OTL-patterns, research traditions signifi cantly explained variance in DIF. While psychology students were more likely to correctly answer items addressing quantitative methods than students with the same overall ability level but from diff erent disciplines, students of all other disciplines were more likely to solve items addressing qualitative methods. These diff erences coincided with diff erences in OTL. Overall, the fi ndings suggest that research competencies are similar across the social sciences, but diff erences between disciplines exist in their focus on quantitative or qualitative methods.