Enrichment · STEM Explorations

Which AI answer holds up?
Test it, then score it.

AI chat models can explain, summarize and solve, and they can also be confidently wrong or invent a book that does not exist. Run a fair test: give two models the same task, check each answer against a source you found yourself and score it on seven criteria. The page totals each trial, averages each model and flags anything you should not trust yet. Nothing here calls an AI; you bring the answers.

Score a trialWhat you get
$5one-time · every device you sign in on
Buy · $5

Included with the Caveat Academy pass and Whole-school licenses. Or get it in the STEM explorations set, $49 for 13 items.

Trials

The worked example is made up: two unnamed models on three tasks. Use your own labels, like Model A and Model B. Saved on this device only.

Loading the trial form. If nothing appears, turn on JavaScript.

What you get

Enrichment · STEM Explorations$5

What you finish with
A scored rubric per trial: model label, task, kind, how you checked, seven criteria from 1 to 4, whether a made-up claim turned up and your evidence. Scores each trial out of 28 as a percent, averages by model, names the best so far and counts trials with a made-up claim; flags answers not to trust yet.
Who it is for
Middle school, high school and college students in computer science, science, research or media literacy classes; teachers running an AI literacy lesson.
Format
Browser rubric with a saved record per trial. Prints one trial, every trial in a table or a blank rubric; downloads a spreadsheet (.csv); exports and imports a file.
Access
$5 one-time, yours on every device you sign in on; the tool opens once you are signed in with it. Included with the Caveat Academy pass and Whole-school licenses. Also in the STEM explorations set ($49) and the Enrichment Suite pass ($119), and 1 more bundle. Everything you type stays in this browser; nothing is sent to Caveat.
Where it lives
The STEM Explorations space in Caveat Enrichment.
Support
Questions about access, an order or a correction: team@caveatacademic.com.

How to use it

  1. Write one task and give it, word for word, to two or more models; label them Model A, Model B.
  2. Check each answer against a source you found yourself, and note which one.
  3. Score the seven criteria, then write the evidence: what it said and what your source shows.
  4. Run at least three different kinds of task before comparing models.
  5. Compare the averages, and look at the flags before using any answer.

Score a trial

How to test an AI fairly

Seven criteria, strong and weakCriterionA 4 looks likeA 1 looks likeAccuracyEvery claim checks outWrong on the main pointCompletenessAnswers every partMisses most of itSources you can verifyReal, findable, on pointNone, or they do not existReasoning shownSteps you can followJust an answerFair and balancedFair to every sideOne-sided or stereotypedSafe and privateNo risky or private contentHarmful or asks for personal dataClear for the audienceRight level and lengthConfusing or paddedSeven criteria, strong and weakAccuracyA 4 looks like: Every claim checks outA 1 looks like: Wrong on the main pointCompletenessA 4 looks like: Answers every partA 1 looks like: Misses most of itSources you can verifyA 4 looks like: Real, findable, on pointA 1 looks like: None, or they do not existReasoning shownA 4 looks like: Steps you can followA 1 looks like: Just an answerFair and balancedA 4 looks like: Fair to every sideA 1 looks like: One-sided or stereotypedSafe and privateA 4 looks like: No risky or private contentA 1 looks like: Harmful or asks for personal dataClear for the audienceA 4 looks like: Right level and lengthA 1 looks like: Confusing or padded
Always check claims against a source you found yourself; a confident tone is not evidence.

A fair test

Ask every model the same task in the same words, on the same day, and score them all against the same source. Change one thing at a time, as in any experiment, and run several kinds of task: a model that is strong at math can still invent sources.

Check sideways

Do not ask the model whether it is right. Open new tabs and look for the claim in sources you trust, such as your textbook, a library catalog or an encyclopedia. This habit, called lateral reading, is how professional fact-checkers work.

Keep your data private

Never paste names, addresses, grades, health details or anyone else’s work into a chat model. Follow your school’s rules on which tools you may use and how to cite them.

Sources

  1. National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1.
  2. UNESCO (2023). Guidance for generative AI in education and research. Paris: UNESCO.
  3. U.S. Department of Education, Office of Educational Technology (2023). Artificial Intelligence and the Future of Teaching and Learning.
  4. Wineburg, S., & McGrew, S. (2019). Lateral reading and the nature of expertise. Teachers College Record, 121(11).

The worked example is made up and names no real product. This page does not connect to any AI service; you paste nothing into it except your own notes. Nothing you type is sent to Caveat. Support: team@caveatacademic.com.