Teaching · For Teachers · Assessment

Which questions
tell strong and
weak apart?

Paste each student’s right and wrong answers. The calculator compares how the top and bottom of the class did on every question and flags the items that fail to separate them, or that point the wrong way and may have a bad key.

Check the itemsWhat you get
$12one-time · every device you sign in on
Buy · $12

Included with Caveat Classroom and Whole-school licenses. Or get it in the For Teachers toolkit, $79 for 72 items.

The item discrimination calculator

The worked example is made up: 15 students on a six-item cell biology quiz. Use rows without names. Nothing you type leaves this browser.

Loading the calculator. If nothing appears, turn on JavaScript.

What you get

Teaching · For Teachers$12 · included with Classroom and Whole-school licenses

What you finish with
Each item’s proportion right in the upper and lower groups, its discrimination index and difficulty, a rating, and flags for weak or negative items. It prints as a summary and downloads as a spreadsheet.
Who it is for
Teachers and departments writing common assessments, assessment coordinators and instructional leaders reviewing benchmark items.
Format
Browser calculator for a phone, tablet or computer, by touch or keyboard. Up to 2,000 students and 200 items. Prints a summary; downloads a spreadsheet (.csv); export and import a file.
Access
$12 one-time, yours on every device you sign in on; the tool opens once you are signed in with it. Included with Caveat Classroom and Whole-school licenses. Also in the For Teachers toolkit ($79) and the Teaching Suite pass ($199), and 1 more bundle. Everything you type stays in this browser; nothing is sent to Caveat.
Where it lives
The For Teachers space in Caveat Teaching.
Support
Questions, a correction or a request: team@caveatacademic.com.

How to use it

  1. Name the test.
  2. Paste a row of 1s and 0s for each student.
  3. Keep 27% groups or choose another size.
  4. Read D and the rating for each item, then check flagged items.
  5. Print the summary or download the spreadsheet.

Check the items

How D is worked out

Top group against bottom group.

The worked example: item 3Rank by total15 studentshighest to lowestTop and bottom 427% eachround(0.27 × 15)Item 3 right3.0 of 4 vs 1.0 of 4upper vs lowerD for item 30.500.75 − 0.25The worked example: item 3Rank by total15 studentshighest to lowestTop and bottom 427% eachround(0.27 × 15)Item 3 right3.0 of 4 vs 1.0 of 4upper vs lowerD for item 30.500.75 − 0.25
An item discriminates when the students who did best on the whole test get it right more often than those who did worst.

The index

Rank students by total score and take the top and bottom groups, classically 27 percent each, a size Kelley showed balances group size against how far apart the groups are. An item’s discrimination index is the proportion of the top group who got it right minus the proportion of the bottom group who did. It runs from −1 to 1; most good classroom items land above .30.

Reading the flags

A low D means the item does not separate students who know the material from those who do not, often because of unclear wording, a trick, or content that was not taught. A negative D, where weaker students do better, usually signals a wrong answer key or an attractive distractor. Items nearly everyone gets right cannot discriminate, which is fine for a mastery check.

Sources

  1. Ebel, R. L., and Frisbie, D. A. (1991). Essentials of Educational Measurement (5th ed.). Prentice Hall: item discrimination and the rating guide.
  2. Kelley, T. L. (1939). The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, 30(1), 17–24.
  3. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing, chapter 4: test design and development.
  4. Crocker, L., and Algina, J. (2008). Introduction to Classical and Modern Test Theory. Cengage: item difficulty and discrimination.

The worked example is made up. Paste scores without names. Your scores stay in this browser. Support: team@caveatacademic.com.