How Much Evidence Is Enough to Judge Mastery?

Jordan aces the first quiz on solving equations. 92 out of 100. It's tempting to mark him "Meets" and move on.

Then the unit test comes back. 64.

So which one is the real Jordan? The truth is, you can't tell yet. And that's the whole point of this post.

In standards-based grading, a level like "Meets" or "3" is a claim. It says: this student can do this skill, reliably. A claim like that needs proof. This guide covers how much proof is enough, what counts, and what to do when you don't have it yet.

The short answer: for most standards, plan on at least three solid pieces of evidence, from more than one kind of task, spread over time, before you give a level. Collect more when the pieces disagree, when a student sits on the line between two levels, or when the decision really matters. Three is a starting point, not a finish line.

What does the research say about how much evidence you need?

No study proves that three is the magic number. But a lot of research agrees on one point: one task tells you far less than you think.

Here are the findings that matter most for classroom teachers.

One task mostly measures the task. In 1993, Richard Shavelson, Gail Baxter and Xiaohong Gao studied hands-on science and math assessments with elementary students. The study found that the biggest source of error wasn't the grader. It was the task itself. A student who did well on one task often did worse on the next, and the other way round. To get a dependable score for one student, they estimated you'd need around 8 to 23 tasks, depending on the assessment. That sounds like a lot because it is. These were broad tests of "science" and "math". A single classroom standard is much narrower, which is why classroom teachers can get away with fewer pieces. But the lesson holds: one task isn't a full picture.

Writing is especially jumpy. A 2015 study in Language Testing by Renske Bouwer and colleagues looked at students' writing scores across different tasks and kinds of writing. They found that only about 10% of the differences in scores came from real differences in students' writing skill. The rest came from which task students got, which kind of writing it was, and who scored it. Their advice for fair comparisons was that students should write at least three texts in each kind of writing, each scored by at least two people.

Even good tests put students in the wrong level. Dylan Wiliam, a leading researcher on classroom assessment, worked out what happens when a test that's fairly reliable sorts students into levels. In his 2001 article "Reliability, validity, and all that jazz", he showed that a test with a reliability of 0.80 (good by most standards) would put roughly 19% to 43% of students in the wrong level, depending on the age group tested. He also pointed out that the only real ways to make a test more reliable are to make it longer, or to aim more questions at the same skill. In other words: more evidence.

Teachers don't grade the same way either. Over a hundred years ago, Daniel Starch and Edward Elliott sent the same English papers to many high school teachers and got back a wide spread of marks. Thomas Guskey and Susan Brookhart, looking back on a century of this research in Educational Leadership, say a big cause is teachers stressing different criteria. Their advice for students near the line between two grades: look at more evidence of learning before you decide.

Put those together and the message is simple. A single score is a snapshot. A level should be a pattern.

Where does the "rule of three" come from?

Many schools that use standards-based grading set a floor of three pieces of evidence per standard. College Community School District in Iowa, for example, writes it into its standards-based learning handbook: "A body of evidence with a minimum of three (3) pieces of evidence is collected for each reporting standard." It adds, bluntly: "One piece of evidence does not constitute a body of evidence." Students who don't complete enough evidence get an "I" (incomplete) instead of a level.

Ontario's province-wide assessment policy, Growing Success, takes a related approach. It says evidence "is collected over time from three different sources": observations, conversations and student products. Teachers call this triangulation. And grading expert Ken O'Connor, in his book A Repair Kit for Grading: 15 Fixes for Broken Grades, recommends recording "insufficient evidence" rather than guessing when the evidence isn't there.

So why three, and not two or five? Three isn't a scientific cut-off. It's the smallest number that can show you a pattern:

• One piece is a single moment. Maybe a great day, maybe a bad one.
• Two pieces that agree are encouraging. Two that disagree leave you stuck. Which one do you believe?
• Three pieces let you see a pattern. If two agree and one doesn't, the odd one out is a question to look into, not the answer.

Think of it like a weather forecast. One sunny morning doesn't tell you it's summer. A week of them does.

Watching the picture come together

Let's go back to Jordan. Here's how his evidence on one math standard, 8.EE.C.7 (solve linear equations in one variable), builds over a few weeks. We'll use a common four-level scale where 75% and up is "Meets" and 90% and up is "Exceeds".

After piece 1, Jordan looked like he'd gone beyond the standard. After piece 2, he looked like he was still getting there. Neither was true. It took the third and fourth pieces to show where he really is.

Notice what happened at piece 3. The evidence didn't just add a number. It raised a question: why did the test go so badly? Maybe he rushed. Maybe the test mixed in a skill he hadn't learned yet. That's the real value of a body of evidence. It shows you the moments that don't fit, so you can find out why.

What makes a piece of evidence count?

Three weak pieces of evidence aren't better than one strong one. Before you count anything toward a level, ask five questions about it. This is what separates good standards-based assessment from just collecting scores.

1. Does it measure this standard? A lab report might touch five standards. If the rubric has a clear row for the one you're judging, that row is evidence. A general "overall quality" score isn't.
2. Is it the student's own work? Group projects are great for learning. But a group score tells you what the group can do. For a level, you need what this student can do alone.
3. Did it come after teaching? A worksheet from the first day of a unit shows where a student started, not where they are. It's useful, just not as proof of mastery.
4. Were the criteria clear? If you'd struggle to explain why it got a 3 and not a 2, it's weak evidence. A rubric or a clear marking guide fixes this.
5. Is it a different task from the others? Three questions on the same quiz, done on the same day, are closer to one piece of evidence than three. Different tasks on different days tell you much more. That's exactly the task problem the Shavelson study found.

Mix up the kinds of evidence

The strongest bodies of evidence use more than one kind of task. For a math standard, that might be a quiz, a problem the student solves and explains in writing, and a quick one-on-one where they talk you through a problem. For a science standard, a lab report, a short test and an explanation of a diagram.

This is the idea behind triangulation. A student who freezes on tests can show you the skill in a conversation. A student who talks a good game can show you whether they can do it on paper. Each kind of task covers for the weak spots of the others.

It matters even more for English learners, students with learning differences, and anyone who struggles with one particular format. If every piece of evidence is a timed written test, you might be measuring test-taking, not the standard.

Should formative assessments count?

Usually, no. Most standards-based grading systems count summative work (the "show me what you've learned" tasks) and leave formative work out.

Formative work, the practice and the assessments for learning you do along the way, is where students should be free to make mistakes. If every practice task counts, students learn to play it safe, and an early stumble drags down a level they've since reached. The district handbook above says practice assignments "are important for practice but will not factor in as a piece of evidence."

That doesn't mean formative work is useless when you're judging mastery. It tells you when a student is ready for a summative task. And if the summative evidence and the formative work disagree wildly, that's a sign to look closer. For the full difference between the two, see our guide to formative and summative assessments.

Does old evidence still count?

It counts, but recent evidence should count more. Learning grows. A student who couldn't do something in September and can do it in November has learned it. Growing Success says a grade should reflect a student's "most consistent level of achievement, with special consideration given to more recent evidence."

How much more weight recent work gets is a choice about how you combine scores. Our guide on how to calculate mastery compares five common methods, with worked examples.

When is three not enough?

Three is a floor. Here's when to keep collecting:

• The pieces disagree. Jordan's 92, 64 and 79 pointed in three directions. A fourth piece settled it.
• The student is on the line between two levels. If the evidence sits right at the edge of Meets, one more task is fairer than a coin flip. This is exactly what Guskey and Brookhart recommend for borderline cases.
• The standard is broad. "Write arguments to support claims" covers a lot of ground. Remember the writing study: scores jumped around a lot from task to task. Broad skills like writing, reading and problem solving need more pieces than narrow ones.
• The decision is a big one. An end-of-year level, a placement decision or a report card that goes home deserves more evidence than a mid-unit check.
• The evidence is old. If the last piece is from two months ago, it may not show where the student is now.

And when can fewer be fine? For a narrow, clear-cut skill, like labeling the parts of a cell, two strong pieces that agree may be enough. You know your standards. The point is to decide on purpose, not by accident.

What should you do when the evidence is thin?

Sometimes a student hasn't done enough work on a standard by the time you need a level. They were absent, the unit ran short, or the standard only came up once. Here's what works:

1. Say so. Write "insufficient evidence" or "I" instead of guessing. It's honest, and it tells students and families exactly what's missing. A guessed level is worse than no level, because nobody can tell it's a guess.
2. Plan one more chance. A short, focused task, like three problems or a paragraph, is often all you need. It doesn't have to be a full test.
3. Have a conversation. Sit with the student for two minutes and ask them to talk you through a problem. That's real evidence, and it's often the fastest kind.
4. Don't fill the gap with a zero. A zero for missing work isn't evidence that a student can't do the skill. It's evidence the work is missing. Mixing those up makes the level mean less.

Five mistakes to avoid

1. Letting one score decide. A single test, however big, is still one piece of evidence.
2. Counting practice as proof. Early practice shows learning in progress. Count it, and you punish students for learning.
3. Counting the same thing three times. Three questions from one quiz aren't three pieces of evidence. Spread evidence across tasks and days.
4. Using only one kind of task. If every piece is a timed test, you're partly measuring test-taking.
5. Hiding the gaps. If you don't have enough evidence, say so. A level with nothing behind it is the first thing a parent will question at a conference.

Setting a minimum evidence rule in GradingPal

Keeping count of evidence for 120 students and 30 standards is the hard part. If you grade in GradingPal, the standards mastery gradebook keeps count for you as you go. Here's how it handles everything above:

• You set the minimum. In each class's standards settings, you choose how much evidence a student needs before they get a level. It's 3 by default, and you can set it anywhere from 1 to 10.
• No guesses before then. Until a student reaches the minimum, the gradebook shows "Insufficient evidence (2 of 3)" instead of a level. You can see at a glance who needs one more chance.
• Practice stays out unless you say otherwise. By default, summative work counts toward mastery and formative work doesn't. Practice still shows up in the student's evidence trail. It just doesn't move the level.
• Weights count toward the minimum. If you set a test to count twice as much as a quiz, it also counts twice toward the minimum. As the setting itself explains, "one graded item in a 2× category counts as 2 points." So if you want three separate tasks no matter what, keep an eye on your weights.
• You can see what's behind every level. Under each student's result, a line shows how it was worked out, for example "7 weighted evidence points · minimum 3". Click any point and the actual graded work opens, with the assignment and item it came from.
• A quick tile shows the gaps. The class view has an "Insufficient evidence" tile that counts every student result still below the minimum. It's a handy list for planning reassessments.
• A conversation can count too. If you check a skill one-on-one, you can set the student's level yourself with a reason, like "Re-assessed one-on-one after the exam." It shows as "set by teacher", and the original evidence stays in place.

The evidence itself comes from grading you're already doing: essays, worksheets, quizzes, exams and lab reports, typed or handwritten. AI suggests which standards a question or rubric row matches, and you approve each one. The mastery math itself is plain arithmetic, with no AI involved.

The bottom line

A level in standards-based grading is a claim about what a student can do. One score can't carry that claim. A pattern can.

So set a floor, usually three pieces of evidence. Make each piece count: aligned to the standard, the student's own work, done after teaching. Mix up the kinds of tasks. Collect more when the evidence disagrees or the stakes are high. And when you don't have enough, say so.

Want the counting done for you? Start free with GradingPal, turn on the standards mastery gradebook for one class, and set your own minimum evidence rule.