Best practice: Asking questions that drive program effectiveness
Once a district has a good handle on what data will be collected, and how it will be stored, accessed, and used, the next critical step in turning data into information is asking the right questions. The MTSS framework provides an important structure for identifying these vital questions with answers to inform next steps for improved practice. In an MTSS framework, we begin with tier 1 or core instruction. This programming must be designed to meet the instructional needs of at least 80% of our students. Said another way, at least 80% of students should meet grade-level standards across subjects and domains given core instruction alone. Using available data, districts can ask critical questions about the extent to which their current core instructional systems are meeting this goal.
Important questions to ask about core instruction effectiveness:
-
What percentage of students in each grade level met our target expectations for fall, winter, and spring in each domain?
-
How do these percent-above-target scores for the current year compare to those of previous years?
-
What percent of students in each grade level who started the year on target finished the year on target?
-
What percent of students in each grade level met target expectations in the spring without receiving supplemental intervention?
-
What percent of students who started the year below target ended the year above target without receiving supplemental intervention?
-
What percent of students who started the year on target made expected growth from fall to spring?
In addition to evaluating core program effectiveness, districts should also regularly review outcomes for students participating in tier 2 and 3 supplemental intervention programs. This enables districts to measure the performance of these interventions to determine if they are getting the expected return on investment.
Important questions to ask about supplemental intervention effectiveness:
-
Are we using intervention programs that have empirical support?
-
How many students are assigned to each intervention that we offer?
-
What is the typical duration of intervention participation for students?
-
Are the students assigned to these interventions receiving a large enough dose to expect to see improvement?
-
What percent of students receiving the intervention are making sufficient progress to catch up?
-
What is the average growth rate for students who participated in “X” intervention?
Many districts look to their data-management platforms to go beyond the necessary data aggregation. They also support users’ ability to evaluate the extent to which the current tier 1/core and current tiers 2 and 3 interventions for instructional programming are meeting expected levels of effectiveness. Data systems that offer clear reporting will enhance district ability to act on results and benefit students. This is also where a data management system that allows users to dig into their data — through multi-select filtering and disaggregation — is important. Schools can investigate patterns within their data to identify subgroups of students for whom the instruction has not been effective and better understand how to respond.