Why Schools Need to Change

Data, Metrics, and the Real Goal: Increased Learning

I am torn between two worlds in which I live: the data world and the innovation world. It has been bothering me for some time. As the director of the Office of Innovation for Education, I’ve spent the past seven years exploring and enabling innovation in education in Arkansas, along with a talented team of learning design and innovation leaders. I also run analytics and modeling for assessment, accountability, and reporting. These two worlds may seem very different but they give me an inside view on how data are informing decisions for learning and that leads me to my current crux.

Over the years I have visited numerous innovative schools that focus on helping learners figure out how they learn best and designing experiences with learners to help them take charge of their learning. When I am in those schools, I don’t see data walls or data journals, I don’t hear about test scores, and I don’t hear students worrying about getting a particular letter grade. Instead, I see learning progressions in student-friendly language posted on the walls of the classroom or held in a personalized learning mapbook. These learning progressions make the ‘what’ of learning visible to the learners. I see students point to where they are on progressions and work with a friend to figure out the next steps. I hear teachers and leaders talking about assessment as an ongoing process rather than an isolated event. In these schools, assessment is used to figure out where learners are in their learning progressions and what next steps they will take. These schools focus on student learning as the most important outcome and they trust that this will show up in their metrics.

In my data world it can be quite the opposite. Conversations in the data-centered schools I have visited focus on metrics: Who is close to the next readiness level? How many points do we need each learner to gain on the test to get to the next level? Who is low growth and by how much? How many students moving from moderate risk to low risk will increase our student engagement score? Teams of teachers spend their collaboration time using these metrics to determine interventions or remediations. Frequently, discussion about strategies to engage students in learning and their own assessment are last on the agenda and get skipped when team collaboration time runs short. While I love the time spent in data analysis, I can’t help feel the disconnection from the learner.

This focus on the metrics has been bothering me lately, a lot! This is the crux of my struggle: I can’t seem to square it with what I am seeing in schools using learner-centered strategies. Their metrics are improving—without conversations focused on the metrics!

Then I came across an article in the September-October 2019 issue of Harvard Business Review, Don't Let Metrics Undermine Your Business? The authors captured my angst and gave it a name—surrogation! It sounds complicated but it is really simple, I promise.

Confusing Metrics with Learning

In business lingo, confusing metrics with strategy is called surrogation. Surrogation is a person’s natural tendency to turn a strategy into its metric! The author’s example, the Wells Fargo debacle, demonstrates how the Surrogation Snare can get your strategy off track. Do you remember all those people who had bank and credit card accounts opened for them without their knowledge, lowering their credit scores? It turns out Wells Fargo employees were focused on a cross-selling metric which they had confused with the strategy—a strategy of building long-term customer relationships, as measured by cross-selling (Harris and Taylor, 2019).

You might be in a Surrogation Snare if you are focusing on increasing survey scores or paying for billboards about student attendance rather than focusing on increasing students’ sense of belonging, connection, and relationships at school. You might be in a Surrogation Snare if you are spending more time talking about achievement and growth scores than you are talking about how to design student learning experiences that support the powerful and deep learning necessary for each student to grow.

Leading with Learning

At the Office of Innovation for Education, a team of learning design and innovation leaders support educators in Arkansas interested in developing student-focused learning systems. Within their community context, these schools develop plans to innovate their learning systems. We try to support our schools so they can lead with learning to rethink the purpose and use of assessments in their classrooms. While we know their metrics, we also know that to change the metrics they will need to focus on learning. To that end we recently launched a small, professional learning network of teachers and leaders interested in grappling with this tension between test-based accountability pressures that lead to a focus on metrics and designing for deeper, more engaged learning. These professional learning networks are intended to support innovative educators in exploring, experimenting with, and sharing their individual and collective learning around this topic. The Assessment for Learning Project had a significant role in supporting us as we shifted from assessment of learning to student-focused assessment for learning.

The Assessment for Learning Project (ALP) is a learning community that is rethinking assessment for learning, agency, and equity. The Office of Innovation for Education’s aim in participating in the ALP is to co-create an ongoing, cross-state collaboration to support Arkansas educators in providing every student with learning experiences that help them reach their full potential. Through this project, teachers in Arkansas’s Schools of Innovation are able to work on a shared problem of practice; specifically, that students in their classrooms are NOT leading their own learning and assessment.

Our Arkansas educators visited schools leading in assessment for learning practices in Kentucky, Indiana, and Virginia. They connected again with peers in Virginia who had been designing and using student-led assessments to gauge students’ depth of understanding and application of complex skills. As a result, these Arkansas educators shared their learning and first-hand experiences trying out student-led assessments in their schools at our Education Innovation Summit last September.

The Assessment for Learning Project enabled educators in our Schools of Innovation to grow in their understanding of how to change the metrics by focusing on evidence of student learning that informs and engages students as agents of their learning. This new focus is shifting teachers’ conversations from assessment of learning (the metrics) to assessment for learning (the strategy) helping them avoid the Surrogation Snare!

Recognizing the Assessment Data Snare

Is your school or classroom in a Surrogation Snare, confusing data metrics for the learning strategies they are designed to measure? Ask yourself these questions:

• In our team time, do we spend more time in data analysis and assigning students to intervention groups compared to time spent discussing our strategies in response to data analysis and whether those strategies impacted student learning?
• Do our team conversations focus mainly on summative and interim metrics rather than evidence closer to the learner that might inform how to support them in learning?
  • For example, are we discussing assessments that helped us uncover student thinking, possible misconceptions, and their progress through learning progressions so we can design next steps for learning?
• Are we reflecting on our current teaching practices and connecting to or rethinking what we know or what we do as a result of discussions about the metrics?
  • For example, are we ensuring that all learners have an opportunity to engage in and demonstrate their learning in meaningful ways?
• Are our assessment strategies helping us build more equitable learning environments that are impacting the achievement gaps we see in our metrics?
• Are our students involved in their assessment? Do they have opportunities to self-assess? Do they get to process their understanding of where they are in their own learning, set goals, and plan for their next steps?

Don’t confuse the metrics with the strategy! If analyzing the metrics doesn’t lead you to new ideas, new knowledge, new skills, and trying new things to improve student learning, then you might be in a Surrogation Snare. Strategies to improve learning have to connect to actions that impact learning and learners. In doing so, the metrics take care of themselves.

Photo at top: The Arkansas ALP network group on the learning journey to Oakton High School in Fairfax County Public Schools, speaking with a panel of students. (Courtesy of Crystal Beshears)