Which questions did the class get wrong? How to read test results question by question
How to read a multiple-choice test question by question: success rates, which wrong answers students picked, broken questions and what to reteach tomorrow.
In short: For every question, look at the share of students who got it right and at which wrong answer the others picked. One popular wrong answer usually means one idea to explain again tomorrow. When even the strongest students picked the same wrong answer, check the question and the key first. Grade AI counts all of this once the sheets are scanned and saved.
What does reading a test question by question mean?
It means looking at each question across the whole class instead of at each student: how many got it right and how the wrong answers were spread. Specialists call it item analysis. By hand, that counting can take longer than the marking itself.
Grade AI (grade-ai.app) is a phone and web app for teachers that prints multiple-choice tests on paper, one sheet per student, reads the filled-in sheets with the phone camera and does this counting automatically, from the scanned answers, not with AI.
How hard was each question?
The simplest measure of difficulty is the success rate: the share of students who picked the correct answer. As a rule of thumb, above about 80% most of the class knows it, 40 to 80% means the topic is partly there, and below about 40% something did not work: the teaching, the question or the key. Grade AI's "Reteach tomorrow" card only appears for a question below 60%.
A worked example
The table is made-up example data, not a real class: 24 students, a short test on fractions, everyone answered every question. Numbers show how many students picked each answer.
| Question (example) | A | B | C | D | Key | Success rate |
|---|---|---|---|---|---|---|
| 1. Which fraction equals 2/4? | 2 | 19 | 2 | 1 | B | 79% |
| 2. Which is larger: 1/3 or 1/4? (A is 1/3, B is 1/4) | 9 | 13 | 1 | 1 | A | 38% |
| 3. Which fraction is in its simplest form? | 5 | 6 | 7 | 6 | C | 29% |
| 4. What is 1/2 + 1/4? (C is 3/4, D is 2/4) | 3 | 1 | 18 | 2 | D | 8% |
| 5. How many quarters make a whole? | 23 | 0 | 1 | 0 | A | 96% |
- Question 1 is fine.
- Question 2 has one dominant wrong answer: 13 of 24 chose 1/4. That is a shared misconception, "the bigger the bottom number, the bigger the fraction". Reteach this one.
- Question 3 has wrong answers spread almost evenly. The class was probably guessing. The topic needs another pass, or the question was unclear.
- Question 4 looks like a disaster, but 18 of 24 picked C, and 1/2 + 1/4 really is 3/4. The key is wrong, not the class.
- Question 5 is so easy it does not tell students apart. Fine as a warm-up.
What do the wrong answers tell you?
Often more than the success rate. One dominant wrong answer means a shared wrong idea: name it and explain it again. Evenly spread wrong answers mean guessing. A wrong option nobody picks is not doing its job; next time, replace it with a typical mistake, as the item-writing guidelines reviewed by Haladyna, Downing and Rodriguez (2002) recommend. More in How to write good multiple-choice questions.
In Grade AI, open the test and switch to the Questions tab. Under every question you see how many students picked A, B, C and D, as a count and a percentage, with the correct answer marked; sort by "Success rate" to put the hardest first. The Question by question button opens the All questions screen with each question's success rate; tap a question for its "Answer distribution".
How do you spot a broken question?
Suspect the question, not the class, when most of the strongest students picked the same wrong answer, when one wrong answer gets far more votes than the "correct" one (question 4 above), or when two answers could both be right and the students who know the topic split between them.
To check, open the test's Overview. Results are listed from best to worst (with more than 11, the top 5 and bottom 5), and tapping a name shows that student's answers next to the key. If the top students share the same "wrong" letter, reread the question and the key.
When at least 40% of the students who wrote the test (and at least three) picked the same wrong answer on the hardest question, the Overview summary names it and adds: "Either the question is misleading, or the class learned it wrong." In the example, that is question 4.
If AI drafted the question: Grade AI's automatic checks catch two identical answers, not two different answers that are both correct.
What if the answer key was wrong?
Saved results are not re-marked against a changed key: each keeps the key it was graded with, and editing a used test shows "Changes will only apply to new scans. Existing results will remain unchanged." So correct the key, then scan the affected sheets again and save; a new scan of the same student replaces their earlier result.
For a single misread mark, open the student's card and tap Edit next to that test. On the Edit answers screen, tap a circle to change the answer, and the percentage and grade are recalculated. Every question is worth one point; there are no weights.
What should you reteach tomorrow?
The question where many students share one wrong answer. That points to one misconception: start from the wrong answer ("Many of you chose 1/4. Why does it look bigger?"), show it with a picture, and ask one similar question at the end of the lesson.
In Grade AI, the Statistics tab has a "Reteach tomorrow" card. It shows the question with the lowest success rate in the classes and period you filter by, only if that rate is below 60%, with the test it comes from, how many got it wrong, the wrong answer picked most often ("Most picked") and the "Correct answer". Tapping it opens the test. "Other hard questions" lists up to five more.
The card picks by success rate alone. In the example it would show question 4, the one with the wrong key. So read "Most picked" first: if that answer is actually right, fix the key instead.
Who has not written the test yet?
Each test's Overview shows "Wrote the test" with a count such as 22 / 24, then "Did not write" with the names of students on the class list who have no result. Statistics has a "Missing tests" card, and every student card its own "Missing tests" list.
When an absent student writes it later, scan their sheet the same way; if it was lost, reprint it from their student card.
How do you compare versions A and B fairly?
Compare question by question. Question 3 in A is usually not question 3 in B, so Grade AI counts the versions separately: in the Questions tab and on the All questions screen you switch between "Version A" and "Version B", and All questions shows the "Average score" for the selected version. There is no chart that compares A and B.
Check who wrote which version: you decide that per class, and if the strongest students mostly wrote A, A will look easier. Match the questions: if question 5 in B scored 30 points lower than question 5 in A on the same topic, suspect the question. Mind the group size: with 12 students per version, one student moves a success rate by about 8 points. With AI, Grade AI writes separate questions for version B, so difficulty can differ; more in Two versions of a test against copying.
Averages, progress and ranking
The question cards show what to teach; the rest of Statistics shows who needs help: class average over time, "Average by test", "Grade distribution", "Top students", "Need help" and "Unusual drop". Student cards show each average and progress, and Home ranks your top 5 and weakest 5 students once at least 6 students have results. Results work on the phone and the web; scanning needs the Grade AI phone app (iPhone and Android), see How to grade a multiple-choice test with your phone.
Frequently asked questions
What is a good success rate for a test question?
There is no single right number. As a rough guide, on a classroom test the questions that about 40 to 90% of students get right tell you the most. Grade AI shows the "Reteach tomorrow" card only for a question below 60%.
Can Grade AI show which wrong answer students chose?
Yes. For every question it counts how many students picked A, B, C and D, separately for each version, and the "Reteach tomorrow" card names the wrong answer picked most often.
Can I fix the answer key after scanning?
You can edit the test, but saved results keep the key they were graded with. Correct the key, then scan the sheets again and save: a new scan of the same student replaces their earlier result.
Do blank answers count as wrong?
Yes, for the score and the grade. A blank question, two filled circles or a mark too close to call counts as wrong. Unclear marks are flagged, and you can fix a misread with Edit answers. The All questions screen leaves blanks out of its counts.
Does AI analyse my test results?
No. Grade AI uses AI only to draft test questions. Grades come from a fixed formula, and every statistic is counted from the scanned answers.
Try it on your next test.
Set up a class, print the sheets, see how long it takes. Scanning happens on the phone. Building and printing work in the browser too.