Background: A recent stream of literature recognizes the impact of good/poor implementation on the effectiveness of programs. However, implementation is often disregarded in randomized controlled trials (RCTs) because they are run on a small scale. Replicated RCTs, although rare, provide a unique opportunity to study the relevance of implementation for program effectiveness. Objectives: Evaluating the effectiveness of an at-scale professional development program for lower secondary school math teachers through two repeated RCTs. Research Design: The program lasts a full school year and provides innovative methods for teaching math. The evaluation was conducted on two cohorts of teachers in the 2009/10 and 2010/11 school years. The program and RCTs were held at scale. Participating teachers and their classes were followed for 3 years. Impact is estimated by comparing the math scores of treatment and control students. Subjects: The evaluation involved 195 teachers and their 3940 students (first cohort) and 146 teachers and their 2858 students (second cohort). Measures: The key outcome is students’ math achievement, measured through standardized assessment. Results: In the first wave, the program did not impact on students’ achievement, while in the second wave, a positive, persistent, and not insignificant effect was found. After excluding other sources of change, different findings across waves are interpreted in the light of improvements in the program implementation, such as enrollment procedure, teacher collaboration, and integration of innovation in daily teaching. Conclusions: Repeated assessment of interventions already at-scale provides the opportunity to better identify and correct sources of weak implementation, potentially improving effectiveness.
Abbiati, G., Argentin, G., Caputo, A., Pennisi, A. (2022). Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program. EVALUATION REVIEW, 46(5), 578-597 [10.1177/0193841X211055354].
Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program
Argentin, Gianluca;
2022
Abstract
Background: A recent stream of literature recognizes the impact of good/poor implementation on the effectiveness of programs. However, implementation is often disregarded in randomized controlled trials (RCTs) because they are run on a small scale. Replicated RCTs, although rare, provide a unique opportunity to study the relevance of implementation for program effectiveness. Objectives: Evaluating the effectiveness of an at-scale professional development program for lower secondary school math teachers through two repeated RCTs. Research Design: The program lasts a full school year and provides innovative methods for teaching math. The evaluation was conducted on two cohorts of teachers in the 2009/10 and 2010/11 school years. The program and RCTs were held at scale. Participating teachers and their classes were followed for 3 years. Impact is estimated by comparing the math scores of treatment and control students. Subjects: The evaluation involved 195 teachers and their 3940 students (first cohort) and 146 teachers and their 2858 students (second cohort). Measures: The key outcome is students’ math achievement, measured through standardized assessment. Results: In the first wave, the program did not impact on students’ achievement, while in the second wave, a positive, persistent, and not insignificant effect was found. After excluding other sources of change, different findings across waves are interpreted in the light of improvements in the program implementation, such as enrollment procedure, teacher collaboration, and integration of innovation in daily teaching. Conclusions: Repeated assessment of interventions already at-scale provides the opportunity to better identify and correct sources of weak implementation, potentially improving effectiveness.File | Dimensione | Formato | |
---|---|---|---|
Abbiati-2022-Evaluation Review-VoR.pdf
Solo gestori archivio
Tipologia di allegato:
Publisher’s Version (Version of Record, VoR)
Licenza:
Tutti i diritti riservati
Dimensione
530.03 kB
Formato
Adobe PDF
|
530.03 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.