Transitioning to Student-Centered Learning: Evaluating Impact

While working with states to implement innovative student-centered learning policies and programs, ExcelinEd has identified policies and practices that hinder new student-centered learning models. Transitioning to Student-Centered Learning: Policy Solutions for States is a policy brief series dedicated to addressing these challenges.

The Challenge of Evaluating Progress and Impact

All new education programs and initiatives are subject to scrutiny from policymakers, media and parents. However, the level of scrutiny state innovation and pilot programs face is often heightened since these innovative programs are, by definition, untested.

Because most existing state innovation or pilot programs are relatively new, they don’t have a track record of success yet. In fact, many schools and districts participating in these programs are still in the design and planning phase so they don’t have a track record at all. As such, little empirical data is available for states to reference when developing a program or measuring the full impact of innovation and pilot programs on students and their outcomes.

Though challenging, it’s critical that states can evaluate, report and communicate the progress and impact of their innovation and pilot programs.

“States [should] have a plan in place for knowing if or when and how to scale up. States can identify processes for evaluating the design and impact of small-scale pilots to determine what aspects of the innovations are working, under what conditions, and for which students. Conversely, the evaluation could also be designed to identify innovations that do not meet their promise, even after adequate time to adjust implementation to improve results.”

Taken from Achieve’s report Creating Innovation Zones to Advance the Promise of Competency-Based Pathways

Recommendation: Develop a Thoughtful Evaluation Plan

States must understand that the transition to student-centered learning will not happen overnight. As states begin this transition, there are practical steps they can take to ensure they develop thoughtful evaluation plans to support their long-term vision for innovative pilots and programs.

Evaluation Plan Objectives

A thoughtful evaluation plan informs parents, teachers, the community, policymakers and more about what is working and what isn’t. The evaluation plan should:

• Supply local leaders with the information necessary for continuous improvement and building community support.
• Meet legislative expectations by providing the information needed to build continued support and potentially expand the program.
• Identify the issues schools are encountering and the resources needed to overcome them.
• Highlight policy obstacles and potential solutions.
• Provide as much transparency as possible, building trust in the program and evaluation process.

Guiding Questions to Develop a Thoughtful Evaluation Plan

Developing an evaluation plan may seem intimidating, but it really comes down to four questions:

1. What are the goals or desired outcomes of the program?

Most programs have clearly stated outcomes either in legislation or in applications. But if this isn’t the case, goals and desired outcomes should be defined at the outset, so the right data is collected and participating schools have clear guidance.

2. How often will reports be produced?

There may be formal statutory reporting deadlines, but more frequent interim and more informal reports should be included. As soon as possible, establish a timeline with due dates for all involved.

3. Who will perform the evaluation?

Progress reports can be formal reports from the state department of education or selected presentations by school leaders at a state board meeting. However, if there is a specific time frame for implementation, a more formal external analysis would carry more weight.

4. What data will be needed?

States should identify early on what data will need to be collected to accurately evaluate the impact of innovation and pilot programs as they mature.

State summative assessments will be able to provide some of the data necessary, but participating schools should be asked to identify what formative and interim assessments will be used.

State longitudinal data systems must be set up to identify students participating in state programs and for which period they participated. This would make it possible to evaluate the impact of programs on long-term student outcomes such as high school graduation and postsecondary success.

Full participation in data collection and evaluation activities should be made clear on any state program applications.

Indicators to Measure Program Quality

It is important to identify the correct indicators and the timing for the associated data collection to appropriately measure state innovation pilot and program quality. While student outcomes should remain at the core of evaluating the impact of state programs, many of the recommended indicators below are particularly valuable in measuring innovation pilot and program effectiveness when student outcome data is limited.

* All outcomes should always be disaggregated by subgroups.