Examining physics teachers’ formative assessment knowledge: a many-facet Rasch model approach
This study describes a sequential, exploratory, mixed methods approach for the development of an instrument to measure physics teachers’ formative assessment knowledge of force and motion as an important component of their pedagogical content knowledge (Shulman, 1986; 1987; Magnusson et al.,1999). Qualitative data were collected over a two-day period from 14 physics teachers participating in a long-term professional development program. It included semi-structured interviews, revised Content Representation (CoRe) and Pedagogical and Professional-experience Repertoires (PaP-eRs), and assessment artifacts (tests, quizzes, rubrics). This data explored the teachers’ explicit PCK, metacognitive aspects of assessing, assessment approaches, learning progressions, and instructional activities of formative assessment of Newton’s laws of motion. Quantitative data from these same teachers were obtained from the Reformed Teaching Observation Protocol (RTOP) (Sawada & Pilburn, 2000; Sawada et al., 2002) and provided further evidence of formative assessment strategies for force and motion incorporated in the teachers’ practice. Analysis of the qualitative and quantitative themes were combined with empirical, theory-based evidence of formative assessment knowledge, including content knowledge to develop multiple-choice items. Items were reviewed by an expert panel for content validity. The instrument was piloted and revised. The revised Knowledge of Formative Assessment (KFA) instrument was administered to 49 certified physics and physical science teachers in a southeastern state. Many-facet Rasch modeling (MFRM) was used to determine the psychometric properties of the instrument and assuming acceptable psychometric properties, determine other teacher characteristics which affected formative assessment knowledge (years of teaching experience, gender, and participation in the long-term professional development program). The KFA instrument exhibited acceptable psychometric properties. Data also indicated significant differences in teacher subgroup measures related to years of teaching experience, gender and participation in a long-term professional development program. Teacher KFA measures were higher for teachers who had taught between 6-10 years, higher for males than females, and higher for those who had participated in the long-term professional development program. Triangulation with the qualitative data provided validity for the significant differences between the KFA scores and teacher subgroups.