Costs, return and planning

"What will it cost, and is it worth it?" is the question that decides whether your project continues. Vague answers ("AI will save a lot of time") lose the argument, and so do falsely precise ones that fall apart on the first question. This lesson shows how to build a simple, honest cost and value model with code, test how sensitive it is to your assumptions, and design a pilot that produces real evidence. The numbers in the examples are illustrative: replace them with your own.

Watch

AI Foundations: Tokens & PricingCursor · 4 min

What an AI solution costs

Count every kind of cost, not only the model bill:

Model cost per task

Model cost depends on how many tokens each call uses, how many calls a task needs, which model you chose and how much of the prompt can be cached. Prices change, so these are example figures in dollars per million tokens:

import pandas as pd

# Example prices in dollars per million tokens (input, output). Check current prices before relying on them.
PRICES = {"large model": (4.00, 20.00), "mid model": (2.00, 10.00), "small model": (0.10, 0.50)}


def cost_per_call(model, tokens_in, tokens_out, cached_share=0.0, cache_read_factor=0.1):
    """Cached input tokens are billed at a fraction of the normal input price."""
    price_in, price_out = PRICES[model]
    billed_in = tokens_in * ((1 - cached_share) + cached_share * cache_read_factor)
    return (billed_in * price_in + tokens_out * price_out) / 1_000_000


# One support task: two calls (rewrite the question, then answer with sources)
CALLS = [("rewrite", 400, 60), ("answer", 3200, 350)]

rows = []
for model in PRICES:
    plain = sum(cost_per_call(model, i, o) for _, i, o in CALLS)
    cached = sum(cost_per_call(model, i, o, cached_share=0.6 if name == "answer" else 0.0) for name, i, o in CALLS)
    rows.append({"model": model, "per task": f"${plain:.4f}", "with caching": f"${cached:.4f}",
                 "per 10,000 tasks": f"${plain * 10_000:,.0f}"})
print(pd.DataFrame(rows).to_string(index=False))

Two things the table shows. Model choice changes the bill by a large factor, so test whether a smaller model meets your quality bar. And at modest volumes, the model is often the cheaper part: from under a dollar to a few tens of dollars per thousand tasks here, against people time measured in minutes at tens of dollars an hour. That is why the value model below matters more than shaving cents.

A value model with honest assumptions

The value of an assistant is the time or errors it saves, after accounting for the times it fails, the time spent checking it and the people who do not use it. This model makes every assumption explicit:

def monthly_net_benefit(
    tasks_per_month,
    minutes_saved,        # per task, when the assistant succeeds
    rework_minutes,       # extra time lost per task when it fails
    review_minutes,       # time spent checking each output
    success_rate,         # share of used tasks where it helps
    adoption,             # share of tasks where people actually use it
    hourly_cost,          # loaded cost of the people's time
    run_cost,             # model + infrastructure + operation, per month
):
    used = tasks_per_month * adoption
    net_minutes = success_rate * minutes_saved - (1 - success_rate) * rework_minutes - review_minutes
    value = used * net_minutes / 60 * hourly_cost
    return value - run_cost, value


BASE = dict(tasks_per_month=4000, minutes_saved=6.0, rework_minutes=8.0, review_minutes=1.0,
            success_rate=0.85, adoption=0.60, hourly_cost=45.0, run_cost=2500.0)

benefit, value = monthly_net_benefit(**BASE)
print(f"value of time saved: ${value:,.0f} per month, running cost ${BASE['run_cost']:,.0f}, net ${benefit:,.0f} per month")

build_cost = 90_000
print(f"payback on a ${build_cost:,} build: {build_cost / benefit:.1f} months" if benefit > 0 else "never pays back")

On these assumptions the net benefit is about $2,700 a month, so a $90,000 build takes almost three years to pay back, which many organizations would not accept. That is a useful finding, not a failure: it tells you to cut the build cost, raise adoption or find a use with more value before you start.

Read the model as an argument, not a prediction. Each parameter is a claim someone can challenge, and each should be measured in the pilot, not guessed: how many tasks, how many minutes saved, how often it fails, how many people really use it.

Stress-test it

A single number hides how fragile it is. Sensitivity analysis varies the uncertain inputs and shows where the case turns negative:

import pandas as pd

# Reuses monthly_net_benefit and BASE from the previous block.
rows = []
for adoption in (0.3, 0.6, 0.9):
    row = {"adoption": f"{adoption:.0%}"}
    for success in (0.70, 0.85, 0.95):
        benefit, _ = monthly_net_benefit(**{**BASE, "adoption": adoption, "success_rate": success})
        row[f"success {success:.0%}"] = f"${benefit:,.0f}"
    rows.append(row)
print("net benefit per month")
print(pd.DataFrame(rows).to_string(index=False))

print("\nHow the case reacts to one change at a time:")
for name, change in [("review takes 3 min instead of 1", {"review_minutes": 3.0}),
                     ("failures cost 20 min of rework", {"rework_minutes": 20.0}),
                     ("volume is half the estimate", {"tasks_per_month": 2000}),
                     ("running cost doubles", {"run_cost": 5000.0})]:
    benefit, _ = monthly_net_benefit(**{**BASE, **change})
    print(f"  {name:36s} net ${benefit:>8,.0f} per month")

The table makes the real lessons visible. Success rate and adoption dominate, so a feature that is only 70 percent reliable, or that few people use, can lose money even though the average looks fine. Review time matters a lot, because it is paid on every task. The cheapest improvements are often not a better model but a better interface, more reliable tools or a smaller review step.

Value you cannot put in dollars

Some benefits are real and hard to count: faster responses that keep customers, fewer errors that avoid incidents, knowledge spread more evenly. Include them, but say plainly that they are not in the number, and do not let them hide a weak core case. Also count costs of failure: a wrong answer in a regulated setting may cost far more than the time it saved. Estimate an expected loss (probability times impact) and subtract it.

Plan the pilot

A pilot is an experiment designed to answer a question: does this solve the problem, for real users, at acceptable cost and risk? Design it like one:

PILOT PLAN

Question:         <the one thing we want to find out>
Users and scope:  <who, how many, which tasks, which data>
Duration:         <usually 4 to 8 weeks>
Comparison:       <the baseline, or a control group doing the work the old way>
Success criteria: <numbers, written before the pilot: outcome, quality, adoption, risk, cost>
Stop criteria:    <what would make us pause or end it early>
Safeguards:       <human review, limits, monitoring, kill switch, who is told it is AI>
Evidence we will collect: <task logs, time measurements, quality samples, user feedback, incidents>
Decision:         <who decides, when, and what happens for each outcome>

Key points:

• Compare against a baseline. "Users liked it" proves little. "Tickets were answered 31 percent faster than the same team's previous month, with no rise in reopened tickets" proves a lot.
• Choose willing, representative users, not only enthusiasts.
• Measure real work, not demos. Sample tasks and check quality independently.
• Collect negative feedback deliberately: what annoyed people, what they stopped using, where they worked around it.
• Watch adoption, which often decays after the first week.
• Update the value model with what you measured, and re-run the sensitivity table.
• Pre-agree the decision: scale, change, or stop. A pilot that cannot end in "stop" is not a pilot.

Plan the work and the risks

For the project itself, break it into phases with a decision point at the end of each, so you can stop cheaply:

Keep a risk register (responsible-AI course) with owners, and review it at every decision point. Typical project risks: the data is worse than hoped, users do not adopt it, a security or legal review blocks it, a model change shifts behavior, the champion leaves, or the cost per task is higher than modeled.

Resources

Check yourself

1. Build the value model for your chosen problem with honest numbers. Which assumption is the weakest, and how would you measure it in a pilot?
2. Run the sensitivity table. At what success rate or adoption does the case turn negative?
3. Write a pilot plan for your project, with the success criteria and the stop criteria.