Partner Spotlight: How Rocketship Education Used Schoolzilla to Get the Most Out of Their Benchmark and Formative Assessment Data

Rocketship Education is a network of K-5 charter schools serving about 6,000 students in California, Wisconsin, and Tennessee (new!). Approximately 90% of Rocketship students are eligible for free or reduced-price lunches. Rocketship has earned national recognition for creating and scaling their innovative school model, which has consistently empowered students from low-income families to realize high levels of academic achievement.

Schoolzilla began partnering with Rocketship in 2013. Since then, we’ve developed a collaborative relationship with the members of Rocketship’s staff who are responsible for empowering teachers with the data they need to fulfill Rocketship’s mission. This Partner Spotlight was written in collaboration with Jan Faraguna, Rocketship’s Director of Analytics, to learn how she and her team have used Schoolzilla to solve real problems for their teachers.

Case study

The problem: Rocketship’s culture of academic achievement is based on helping teachers respond to their students’ needs as quickly as possible. Technology and regular assessments play a role, but the core of instruction is still the interactions between teachers and students. To help teachers target their instruction more effectively, Rocketship strives to empower teachers with data. Faraguna explained that before Schoolzilla, “teachers were going to many, many different places for their data, and they were spending many hours doing a lot of analysis on their own.”

One specific problem was that benchmark assessment data were often not granular enough to help teachers make student-level instructional decisions. Rocketship’s Data Team found that they were spending an inordinate amount of time after each assessment cycle just constructing data sets for analysis. “It could take so much time just to do surface-level analyses that weren’t that insightful,” says Faraguna.

The solution: Jan and her team observed school leaders and teachers during regularly scheduled “data days” and engaged with them at focus groups to identify their exact needs. Leveraging Schoolzilla’s data model, they constructed teacher-specific reports for Rocketship’s benchmark and NWEA MAP assessments to provide teachers fast access to item, standard, and strand-level analysis for every student.

Anatomy of a Teacher Benchmark Report

Picture 1 is a screenshot of a report that helps teachers analyze benchmark results. We’ve added green boxes with the numbers 1-4 to illustrate the flow that teachers can go through in order to gain granular insight into their students’ successes and struggles. A teacher begins by filtering for their school, subject, grade level, and class using the dropdown menus on the left side of the screen. In box  1 , teachers can see their class average performance in each subject strand for every test cycle. You can see by the red and green color-coded boxes that this class performed much better on average during testing cycle 4 than in the previous testing cycles.

Picture 1 – Teacher Benchmark Report

teacher-benchmark-report

Each strand in this benchmark covers several individual standards. By clicking on a specific strand in box  1  (in this example, “Literary Response & Analysis” has been selected) teachers can drill down to see class-wide average performance on each standard and for each assessment cycle (displayed in box  2 ).

Class averages are a helpful starting point, but the most actionable data connects specific students and specific content. Each standard is assessed using several test items, which allows teachers to drill down further to see the percentage of their class that answered each individual item correctly (displayed in box  3 ).

For example, you can see in box  3  that students scored very differently on the two items covering standard 3.2. Teachers typically want to see the differences between the questions that most students answered correctly and incorrectly in order to diagnose the source of students’ misunderstanding. To do that, teachers can drill down once more to see each individual student’s actual response to each item (student names are blurred in this screenshot to protect anonymity). The responses are highlighted in red when incorrect, and in green when correct (displayed in box  4 ).

To go even further, teachers have the ability to see the distribution of answers that their students gave to each test question. Picture 2 is a screenshot of a related report that shows the same class average results for the “Literary Response & Analysis” strand (displayed in box  1 ). You can see that only 43% of this teacher’s students answered item 25 correctly.

Picture 2 – Item Breakdown

NWEA-MAP-item-breakdown

By clicking on “43%,” teachers can drill down to see the exact distribution of letter answers that their students submitted on that test item (displayed in box  2 ). The correct letter answer is written in green (in this case, the correct answer was “C”). The teacher can immediately tell that 30% of his/her students answered “D,” and 20% answered “A.” Overall, this type of standard and item-level analysis helps teachers identify areas for corrective instruction and unit planning.

Anatomy of a School-wide NWEA MAP Report

NWEA MAP is a key assessment for many school systems because it measures growth during and across school years, and it’s nationally normed. Many schools, including Rocketship’s, have explicit goals for both growth and absolute performance on this assessment. In fact, we modeled some of the reports in Schoolzilla’s NWEA MAP Explorer after Rocketship’s work. In Picture 3 (below), we highlight a custom report that Rocketship built to track progress toward goals and identify strengths and challenges.

Picture 3 – School-wide NWEA MAP Report

Schoolwide-NWEA-MAP-report

As before, teachers and school leaders begin at box  1  by filtering for their year, subject, school site, and testing cycle. The color coding in Picture 1 shows the percentage of Rocketship’s students whose scores fall into each of NWEA’s four proficiency categories (written in the report legend as “quartiles”). For example, the column selected at box  2  shows that 63% of Rocketship’s 4th graders exceeded proficiency in Mathematics at the spring assessment cycle in 2014.

Jan’s team structured this report so that school leaders can see their students’ performance distribution over time. Clicking on the spring performance column (at box  2 ) allows school staff to reveal the performance distributions for all grades in all years on their spring assessment cycles (displayed in box  3 ). In this example, you can see that the 4th graders have made substantial progress since fall of the present year, and they’re outperforming most other grades. Box  3  shows that these 4th graders have also outperformed 4th-grade classes from the previous two school years.

Partnering with Schoolzilla

Rocketship has high expectations for its students, staff, and partners. Jan and her team aren’t afraid to push Schoolzilla to develop and improve in ways that will benefit Rocketship’s staff and students. Rocketship is constantly evolving their analytic tools, which means they need systems that will evolve with them. “Tableau and Schoolzilla in general are pretty flexible,” says Faraguna, “and we are able to respond to feedback and change the reports to be more user-friendly for teachers.”

Like many, Jan and her team developed considerable Excel skills before partnering with Schoolzilla. When asked to compare the process of data analysis and reporting before Schoolzilla, Faraguna notes that, with Schoolzilla, “the usability is much easier, and we’re able to do more cuts of the data for school staff. [Schoolzilla] is just much more visual and interactive than Excel.”

Impact

Jan says that the results of Rocketship’s partnership with Schoolzilla can be felt at all levels of their school network. “Now that [the data] is there and it’s available for everyone,” she says, “[teachers] are able to really dive in and not spend time analyzing, but spend time interpreting and using data to plan. We had much more meaningful conversations in the very beginning part of the year because the data was so readily available.” School leaders appreciate being able to have data-informed instructional conversations without having to spend time preparing the analyses.

Further, Jan says that Schoolzilla opens up her team’s capacity “to go deeper into the things we want to understand because we’re not spending all of our time creating data sets and doing basic analyses.”

Schoolzilla is proud to partner with Rocketship Education, and we look forward to continuing to support their efforts to provide a world-class education for their students.

CTA