Teaching · For Teachers · Assessment

Same test, twice.
Do the scores
hold steady?

Paste the scores from two sittings of the same test. The calculator gives the reliability coefficient with an honest interval around it, shows whether the whole group shifted, and points to the students whose scores moved most.

Check the reliabilityWhat you get
$12one-time · every device you sign in on
Buy · $12

Included with Caveat Classroom and Whole-school licenses. Or get it in the For Teachers toolkit, $79 for 72 items.

The test-retest calculator

The worked example is made up: twelve students on a spelling inventory given twice, two weeks apart. Use codes, not names. Nothing you type leaves this browser.

Loading the calculator. If nothing appears, turn on JavaScript.

What you get

Teaching · For Teachers$12 · included with Classroom and Whole-school licenses

What you finish with
The test-retest coefficient with a 95% interval, the mean change, the implied standard error of measurement, a scatter plot and the students who moved most. It prints as a summary and downloads as a spreadsheet.
Who it is for
Teachers and departments checking a common assessment, school psychologists, assessment coordinators and graduate students in education.
Format
Browser calculator for a phone, tablet or computer, by touch or keyboard. Up to 2,000 pairs. Prints a summary; downloads a spreadsheet (.csv); export and import a file.
Access
$12 one-time, yours on every device you sign in on; the tool opens once you are signed in with it. Included with Caveat Classroom and Whole-school licenses. Also in the For Teachers toolkit ($79) and the Teaching Suite pass ($199), and 1 more bundle. Everything you type stays in this browser; nothing is sent to Caveat.
Where it lives
The For Teachers space in Caveat Teaching.
Support
Questions, a correction or a request: team@caveatacademic.com.

How to use it

  1. Name the test and type the days between sittings.
  2. Paste the first scores, then the second scores in the same order.
  3. Read r and its interval, then the mean change.
  4. Look at the students who moved most for reasons outside the test.
  5. Print the summary or download the spreadsheet.

Check the reliability

How the coefficient is worked out

Correlation between two sittings.

The worked example, from pairs to reliabilityPair the scores12 studentssame order both timesCorrelater = 0.950PearsonFisher z interval0.83 to 0.9995%, n = 12Implied SEM1.86SD × √(1 − r)The worked example, from pairs to reliabilityPair the scores12 studentssame order both timesCorrelater = 0.950PearsonFisher z interval0.83 to 0.9995%, n = 12Implied SEM1.86SD × √(1 − r)
Students kept nearly the same order across the two sittings, so r is 0.95; with only 12 students the interval around it is wide.

The coefficient

Test-retest reliability is the correlation between the same students’ scores on two sittings. If the test measures something stable, students keep roughly the same order and r is near 1. With small groups the estimate is rough, so the page gives a 95% interval using Fisher’s z transformation. The implied standard error of measurement turns r back into score points.

What affects it

The interval between sittings matters: too short and students remember answers, too long and real learning changes scores. A correlation ignores a shift that is the same for everyone, so a whole class gaining five points can still show high reliability. Testing conditions, fatigue and guessing all lower r.

Sources

  1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing, chapter 2: reliability and precision.
  2. Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507–521.
  3. NIST/SEMATECH e-Handbook of Statistical Methods, correlation and its confidence interval (itl.nist.gov).
  4. Crocker, L., and Algina, J. (2008). Introduction to Classical and Modern Test Theory. Cengage: test-retest reliability and the coefficient of stability.

The worked example is made up. Use student codes, not names. Your scores stay in this browser. Support: team@caveatacademic.com.