Advancement · Data & Impact

Which program buys
more results per dollar.

Enter what two programs cost, including donated resources, how many people each served and how many reached the outcome. The calculator shows cost per participant, cost per result and how sure the ranking is.

Open the calculatorWhat you get
$12one-time · every device you sign in on
Buy · $12

Included with Caveat Classroom and Whole-school licenses. Or get it in the Grant writer’s desk, $49 for 32 items.

The calculator

The worked example compares made-up small-group tutoring with made-up reading software. Change any number and every result, sentence and chart follows.

Loading the calculator. If nothing appears, turn on JavaScript.

What you get

Advancement · Data & Impact$12

What you finish with
For two programs: cost per participant, cost per outcome achieved, cost per additional outcome over what would happen anyway, the incremental cost of choosing the more effective one, outcomes per budget, and which assumptions could change the answer. Print it or save a spreadsheet file.
Who it is for
Program directors, evaluators, grant writers, school and district leaders, and board members comparing programs or reporting value for money to a funder.
Format
Browser tool. Works on a phone or computer. Print or save as PDF, a .csv of the results, and a file to move figures between devices.
Access
$12 one-time, yours on every device you sign in on; the tool opens once you are signed in with it. Included with Caveat Classroom and Whole-school licenses. Also in the Grant writer’s desk ($49) and the Advancement Suite pass ($99), and 1 more bundle. Figures stay in this browser; nothing is sent to Caveat.
Where it lives
The Data & Impact space in Caveat Advancement.
Support
Questions, a correction or a request: team@caveatacademic.com.

How to use it

  1. Name the outcome you count, such as "reach the spring reading benchmark".
  2. For each program, list every cost for one year (staff, training, materials, space, devices), including donated resources at what they would cost to buy.
  3. Enter the participants served and how many reached the outcome.
  4. Set the comparison rate: how many would reach the outcome without either program, from a comparison group, last year's data or a baseline.
  5. Read the results, then check the sensitivity section to see which numbers you need to firm up.

Open the calculator

How the ratios work

Standard cost-effectiveness arithmetic, written out.

The formulas

  • Total cost is the sum of every ingredient for the same period, including donated time, space and goods valued at market prices. Leaving out donated resources makes a program look cheaper than it would be to copy.
  • Cost per participant = total cost ÷ participants served. Cost per outcome achieved = total cost ÷ participants who reached the outcome.
  • Additional outcomes = participants who reached it − participants × comparison rate: the results beyond what would have happened anyway.
  • Cost-effectiveness ratio = total cost ÷ additional outcomes, which equals cost per participant ÷ effect per participant (success rate − comparison rate). Lower is better. It is undefined when a program does no better than the comparison rate.
  • Incremental cost-effectiveness ratio = (cost per participant of A − of B) ÷ (effect of A − effect of B): what each extra success costs if you choose the more effective, more expensive program. If one program is both cheaper and more effective, it dominates and no ratio is needed.

Reading the results with care

  • The ratios are only as good as the comparison rate. Without a credible comparison group, treat the result as descriptive, not as the effect of the program.
  • Compare programs only on the same outcome, measured the same way, over the same period, with participants who are alike.
  • Sensitivity: each assumption moves up and down by the range you pick while the others stay put (one-way sensitivity). The tornado chart puts the assumption that moves the ratio most at the top. If the ranking flips inside a plausible range, gather better data before deciding.
  • The budget comparison assumes cost grows in proportion to participants. Real programs have fixed costs, so check the scale you would actually run.
  • Programs that run longer than a year should discount future costs and outcomes; this calculator handles one year at a time.

The worked example, checked with exact fractions when the page is built: A costs $96,000 for 80 participants ($1,200 each); 52 reached the benchmark against 24 expected anyway at a 30% comparison rate, so 28 additional outcomes and $3,428.57 per additional outcome. B costs $41,000 for 120 participants ($341.67 each); 54 reached it against 36 expected, so 18 additional and $2,277.78 each. Choosing A over B costs $4,291.67 per extra success, and the ranking flips at a comparison rate of 37.0%. All figures are examples.

Sources

  1. Levin, H. M., McEwan, P. J., Belfield, C. R., Bowden, A. B., and Shand, R. D. (2018). Economic Evaluation in Education: Cost-Effectiveness and Benefit-Cost Analysis (3rd ed.). SAGE. The ingredients method, valuing donated resources, and cost-effectiveness ratios in education.
  2. Drummond, M. F., Sculpher, M. J., Claxton, K., Stoddart, G. L., and Torrance, G. W. (2015). Methods for the Economic Evaluation of Health Care Programmes (4th ed.). Oxford University Press. The incremental cost-effectiveness ratio, dominance and one-way sensitivity analysis.
  3. Hollands, F. M., Kieffer, M. J., Shand, R., Pan, Y., Cheng, H., and Levin, H. M. (2016). Cost-effectiveness analysis of early reading programs: A demonstration with recommendations for future research. Journal of Research on Educational Effectiveness, 9(1), 30 to 53. Comparing reading programs by cost per unit of effect.

A calculator, not an evaluation: it divides the numbers you enter and cannot tell whether a program caused the outcomes. The worked example's programs, costs and results are made up. Support: team@caveatacademic.com.