Measuring Mastery
Choose your metrics, choose your culture.
I had the chance to get back out on the baseball field for the first time in about a decade. I got the same butterflies that I always do walking up to the batter’s box and preparing every pitch for a ball to come my way. What was different this time around was playing with teammates who were half my age and wondering if my body would hold up to stresses of explosive movements that I haven’t pushed myself into in years. Thankfully I was able to enjoy playing under the lights with teammates who I have known for decades, come away with a win, and most importantly, not get hurt. While my body couldn’t react the way it used to when I was in college, my deep understanding of the game and constant drilling with my son’s Little League teams helped me still play the game I love at a high level. It also made me realize what mastery really looks like.
After playing and coaching baseball for over 30 years, having worked with 5-year-olds playing t-ball up through adults playing independent ball (and slow pitch softball), I have the perspective of a wide range of skill levels and progressions in understanding of the multitude of aspects of the game of baseball. I have seen a multitude of different playing styles and countless game situations that I believe I can break down any part of the game into simple parts and explain to anyone regardless of their background or skill level. And yet, I still know there is more to learn about the game of baseball. There are more drills, more ways to use technology, and better data analysis that can improve my understanding of the game.
Mastery comes in many different forms and levels. The fluency around the game of baseball is probably the area I have mastered the most. But, my mastery of baseball is different than a professional baseball player’s mastery. And, it’s different than the types of mastery that students will achieve in class. Mastery for high schoolers is understanding concepts or developing skills to the point where they can successfully start a career. We need to tell high schoolers mastery looks and feels different for them, where they are, in their professional development.
Unfortunately, many schools and classrooms are trying to jam a square peg into a round hole, using Learning Management Systems (LMS) to fit a version of mastery-based grading into a letter grade. We can’t shift the culture of school unless we focus our effort on better ways to communicate what mastery is and capture student progress towards their version of mastery. As long as a letter grade remains the key incentive for students as they pursue college, and for teachers to incentivize their students and to demonstrate efficacy, the education system will optimize the system for those letter grades. Realizing that we need new metrics in education isn’t novel. Coming up with novel ways to help students better understand where they are in their journey and how they can improve is the true innovation we desperately need to develop.
For over a decade, my passion for baseball brought a temptation to replicate the SABRmetrics revolution that transformed the game of baseball (as seen in the movie Moneyball). I thought if we could just get the right data and apply it to teaching and learning in creative ways, we could fix everything in education. I even coined the term Edumetrics and invented new metrics with catchy names thinking it would help clarify what we needed to measure. This was the wrong direction completely. We don’t need more complexity in education. We instead need more focus and simplicity.
Mastery has two key components:
The ability for knowledge or a skill to persist over time.
The ability to apply knowledge or a skill in novel situations.
Inside Project Leo we are capturing both components through student project posts. The projects incorporate the educational goals of the classroom and build on them to allow students to learn how to apply them in real world situations they are interested in, or novel situations. Over the class period, new projects can build on these concepts reinforcing a student’s understanding and helping them to retain their new skills, testing if the skills persist over time as the student develops mastery in these new areas. Student posts will have a community measure of mastery for each knowledge and skill developed in that project.
Behind the scenes we will be tracking the growth of each knowledge and skill over time. Teachers will have access to data visualizations of how each student progresses over time. As students complete their second or third project for a class, we will also be able to see how knowledge and skills are applied by the student in novel situations. The more unique applications will get more attention and feedback from professionals in the community, increasing both student mastery and their professional network. With Project Leo school work can have a direct connection to mastery and career networking, instead of simply an isolated connection to a letter grade.
As we build out Project Leo this summer and beyond, making a streamlined process for students to develop mastery through personalized projects and community feedback is our core focus. To ensure we stay aligned with that focus and are progressing, we have five metrics as our guide:
Increase number of personalized projects built in a classroom that aligns student interests with the knowledge and skills that are highly valued by the current job market.
Increase the number of teachers using Project Leo.
Increase the number of professionals giving feedback to student project posts.
Improve mental health for students and teachers through an increase in agency, purpose (Ikigai), and impact.
Increase social mobility for students by building professional networks through projects built during the school day leading to interactions with professionals.
By focusing on these metrics we hope to build a culture around Project Leo that focuses on giving every student the best possible experience in all of their classes, while supporting teachers to transition into the realm of Artificial Intelligence-enhanced learning. We hope every parent feels confident in supporting their child because they have a clear understanding of what their student is passionate about, while also having a clear mechanism of support by helping build their student’s professional network. We will also continue to build partnerships that unclog the high school to career pipeline with more nimble and relevant classroom experiences that build equitable access to high quality careers.
My mastery of baseball didn’t come overnight, but it developed as I learned its concepts and experienced the game. Likewise, student’s mastery of a subject will be enhanced through their application of a concept in a project that they are interested in. The change in baseball metrics has caused a huge culture shift in what was a very traditional and hard to change industry. At the very least, Project Leo will give new metrics for teachers and schools to consider as they grapple with Artificial Intelligence and a rapidly changing economy. We look forward to supporting teacher and schools during that transition period any way we can.