Do you work in a school district’s central office and have a spreadsheet headache you’d like help with? Schoolzilla has partnered with a funder to offer 25 complimentary 1-hour consultations where you can learn spreadsheet tools and tricks. Spend less time cleaning and organizing data and more time getting actionable insights.
As schools await their Common Core test results, educators, instructional leaders and data analysts across the country have been developing thoughtful ways to understand their first Smarter Balanced Assessment Consortium (SBAC) and Partnership for Assessment of Readiness for College and Careers (PARCC) data.
Schoolzilla reached out to some of these thought leaders to understand how they were planning to approach their Common Core data. This guide is based on the recommendations they shared for how schools can analyze, understand, and act on this new data
1. Define what a “good score” is.
In the first year of a new assessment, you can’t compare your data to historical results. How can you identify the bright spots and growth areas in your data without a baseline of comparison?
First, brace for a drop in proficiency.
States that have already implemented Common Core standards-aligned exams have found that student proficiency rates have dropped significantly. Most states are expecting similar performance declines on Smarter Balanced and PARCC tests due to more challenging content and more demanding test questions. Based on the cut scores Smarter Balanced approved last fall, only 33 percent of students are projected to reach the proficiency mark in 11th grade math.
This drop in proficiency means that it’s hard to interpret your scores. Lower scores could mean less student learning; or, they could just reflect a change in the achievement measuring stick. But, in the absence of comparable historical results, there are other ways to put your data in context. You can find other points of comparison that help your families, teachers, and leaders see where your school is making progress.
District, state, and national norms can provide additional measuring sticks for schools. With 18 states administering Smarter Balanced and 11 states administering the PARCC tests, results will allow for more comparisons across states. Smarter Balanced released nationally normed data from its 2014 field test and is expected to do the same with the 2015 results. This data will allow analysts to visualize how their networks, districts, and schools performed relative to national averages.
Look to peers.
In addition to using national norms, instructional leaders told us they plan to compare their performance against high performing schools.
As Elise Darwish, Chief Academic Officer of Aspire Public Schools said, “The first thing I want to do is understand what ‘good’ schools’ data looks like with these new assessments. I’ll be looking at Aspire schools that have been strong in the past, so I can use their scores as a rough benchmark. I’ll also ask other school systems if they’ll share their results so we can compare.”
Compare to other exams.
Analyzing Smarter Balanced and PARCC results alongside other summative assessment sources, such as NWEA MAP and historical results from previous state assessments, will create a more comprehensive, meaningful portrait of student performance.
2. Go deeper.
In addition to snapshots of overall student performance by scale score and achievement level, schools want detailed performance reports to show strengths and weaknesses on particular areas of each test.
Both consortia will report overall scores, as well as performance levels on the particular claims/sub-claims that make up each test. Although data at the question level will not be available at first, claim-level data will help teachers and instructional leaders understand how students did on performance tasks and higher-level content assessed by the Smarter Balanced and PARCC tests.
Looking at claim- or standard-level data for your classrooms or schools is crucial to making your data actionable. Comparing performance across claims can help you find educators who have best practices to share. It can also help set instructional priorities for the next year, instead of feeling like you’re conducting a post-mortem that’s all about last year’s scores.
According to a recent survey conducted by the Public Policy Institute of California, a majority of public school families in California (55%) say they have heard nothing at all about the Smarter Balanced Assessment System. Only eight percent say they have heard a lot about the tests.
Teachers and data analysts we spoke to emphasized their desire to create reports that foster dialogue and help families make sense of their child’s scores on these new tests. Denver Public Schools Parent/Student Portal Manager Juan Pablo Parodi explained: “We did a lot of user interviews with our parents and learned that assessment information is often meaningless to them because of the context in which they are presented with it. We learned that in order for assessments to matter to parents, we needed to present the data in a simple, digestible way that allowed them to quickly grasp whether or not their student was on track, and could inform more constructive conversations with their students and teachers.”
4. Analyze achievement gaps.
One of the most striking findings from the SBAC field test data is the way that it continues to illuminate achievement gaps for students of color and low-income students. These gaps are not new, but it is striking to see them surface repeatedly and dramatically, especially in a wide, nationally representative sample. On the SBAC field test, for example, black fourth graders scored more than six-tenths of a standard deviation below the total group of fourth graders in math. Analyzing how Common Core results differ by race, income level, primary language, and special education classification can help you find and focus on the most critical achievement gaps in your student population.
5. Compare student achievement data with classroom observations.
Common Core standards are designed to encourage educators to make their classrooms more student-centered and their instruction more rigorous. As teaching practice expert Charlotte Danielson told Education Week, “[The Common Core] requires instructional strategies on teachers’ parts that enable students to explore concepts and discuss them with each other, to question and respectfully challenge classmates’ assertions.”
Your results will provide an opportunity to ask a related question: What student and teacher behaviors are present in classrooms where students performed best on Common Core assessments?
Lander Arrieta, a consultant who worked with Duval County on its Common Core implementation, explained, “The first thing I’d want to do is visit classrooms where students scored well and see what those teachers are doing—are they leading student-centered classrooms and facilitating rigorous conversations between students?”
If your district uses a formal evaluation system, you may want reports that compare student results with teacher evaluations on specific instructional competencies. With or without formal observations, walk-throughs with your “bright spot” teachers can help you think about what instructional strategies you want to help everyone on your team develop.
6. Analyze data from computer-adaptive testing.
Computer-adaptive testing creates a number of valuable metrics about a student’s test-taking experience that are unavailable through paper testing. For example, data that may become available from Smarter Balanced includes the time a student spends on a question, as well as the number of times the student changed an answer. Both consortia will also gather data about the types of accommodations available to students, as well as which assistive tools a student actually used during testing.
This data can add insight into each student’s test-taking process, informing an understanding of how the new testing format impacted results, and allowing educators access to evidence about students’ testing stamina and perseverance (or lack thereof) on more rigorous question types. Although this data will not be available initially, future analysis of statistics like testing duration, time per question, and assistive tool usage holds considerable promise for educators and education leaders.
We’d love to hear your thoughts, questions, and concerns about how you plan to analyze your Common Core results as well. Stay tuned for details on how to join the Data Champion Hub—our online community for K-12 data advocates to engage in discussions and share best practices.