LLM Token Billing in Elixir: Metering AI Usage Per Customer

Every pricing page written before 2023 assumed the marginal cost of a request was approximately zero. Storage was cheap, CPU was cheap, and a seat that used the product ten times a day cost you the same as a seat that used it once a month. Then you added an AI feature, and one customer pasted a 400 page PDF into it.

LLM token billing in Elixir is the answer to that: measure what each customer’s AI calls actually cost you in tokens, turn that into money, and either deduct it from a balance they have paid for or add it to their invoice. It sounds like arithmetic. The parts that bite are that you do not know the cost until after the call has finished, that the call can fail halfway, and that your retry logic will happily charge the same customer twice.

This post covers all three, with working code on Aurora Meter, the free MIT metering and entitlements core for Phoenix.

Why per-seat pricing breaks on AI features

Run the numbers on a document summariser. A short email is maybe 800 input tokens and 300 output tokens. At current frontier model prices that is a fraction of a cent. A long contract with a detailed summary is 180,000 input tokens and 4,000 output tokens, which is closer to a dollar. Same feature, same button, same seat, and a cost difference of two orders of magnitude.

Charge a flat $29 per seat and one power user wipes out the margin on nine others. Charge enough to cover the power user and you have priced out everyone else. This is exactly the situation that usage-based pricing exists for, and we have written about choosing between metered and seat-based pricing if you are still deciding. For AI features specifically the decision is usually made for you by the cost curve.

Step one: read the real token counts

The most common mistake is estimating. People reach for a tokenizer, count the prompt, guess the completion length, and bill on the guess. It drifts, and the drift is always in the wrong direction.

Every provider returns the authoritative counts in the response. Read those.

defmodule MyApp.AI do
  @endpoint "https://api.openai.com/v1/chat/completions"

  def complete(messages, opts \\ []) do
    body = %{
      model: Keyword.get(opts, :model, "gpt-4o-mini"),
      messages: messages,
      max_tokens: Keyword.get(opts, :max_tokens, 1_000)
    }

    case Req.post(@endpoint, json: body, auth: {:bearer, api_key()}, receive_timeout: 120_000) do
      {:ok, %{status: 200, body: %{"choices" => choices, "usage" => usage}}} ->
        {:ok,
         %{
           text: choices |> hd() |> get_in(["message", "content"]),
           input_tokens: usage["prompt_tokens"],
           output_tokens: usage["completion_tokens"],
           cached_tokens: get_in(usage, ["prompt_tokens_details", "cached_tokens"]) || 0
         }}

      {:ok, %{status: status, body: body}} ->
        {:error, {:api_error, status, body}}

      {:error, reason} ->
        {:error, reason}
    end
  end

  defp api_key, do: Application.fetch_env!(:my_app, :openai_api_key)
end

Three details worth noticing.

Input and output tokens are priced differently, usually four to eight times apart, so a single total_tokens number throws away the information you need. Keep them separate.

Cached input tokens are billed at a discount by most providers. If you bill the full input count on a cached prompt you are overcharging, and on a chat product with a long system prompt that is most of your traffic.

Streaming responses omit usage by default. Ask for it explicitly, or you will reach the end of a stream with no idea what it cost:

body = %{
  model: "gpt-4o",
  messages: messages,
  stream: true,
  stream_options: %{include_usage: true}
}

The usage object arrives in the final chunk, after the last content delta. Anthropic reports the same thing as message_start and message_delta events carrying input_tokens and output_tokens. Either way, accumulate it and settle at the end of the stream, not at the start.

Step two: turn tokens into money

Prices change, models multiply, and you want a markup. Put all three in one module so there is exactly one place to edit when a provider cuts prices.

Aurora Meter’s ledger uses integer micro-dollars: one dollar is 1,000,000. That is not a style preference. Floats cannot represent 0.1 + 0.2 exactly and a ledger that drifts is worse than no ledger, while cents round a request costing a fortieth of a cent down to zero, which makes a million of them free.

defmodule MyApp.AI.Pricing do
  # micro-dollars per 1,000,000 tokens, i.e. the provider's per-million price in dollars
  @models %{
    "gpt-4o-mini" => %{input: 150_000, output: 600_000, cached: 75_000},
    "gpt-4o" => %{input: 2_500_000, output: 10_000_000, cached: 1_250_000}
  }

  @margin 1.4

  def cost(model, %{input_tokens: inp, output_tokens: out, cached_tokens: cached}) do
    rates = Map.fetch!(@models, model)
    billable_input = inp - cached

    raw =
      billable_input * rates.input +
        cached * rates.cached +
        out * rates.output

    raw
    |> div(1_000_000)
    |> Kernel.*(@margin)
    |> ceil()
  end
end

ceil/1 rather than round/1 is deliberate. Rounding down on millions of small calls is a slow leak, and nobody has ever complained about being charged a hundredth of a cent too much.

A 40% margin on inference is a reasonable starting point: it covers your failed calls, your retries, the tokens you burn on prompt engineering in production, and the fact that the provider will not warn you before a price change. Model it explicitly rather than discovering it in your P&L.

Step three: choose a shape, allowance or prepaid

Two shapes work for AI usage, and they answer a different question.

	Monthly allowance with overage	Prepaid credits
Customer pays	A subscription, plus overage in arrears	Money up front, spent down
You are exposed to	A month of unpaid inference	Nothing
Best for	Predictable usage, established customers	Self-serve signups, expensive per-call work
Fails when	A customer disputes the overage invoice	A customer runs out mid-task

If the marginal call costs you real money before the customer has paid anything, prepaid is the safer default. A stolen card on a self-serve signup with a monthly allowance is a direct transfer from you to an attacker’s inference budget.

The allowance shape

Declare it in the plan DSL. metered gives an included quantity plus a unit price for anything beyond it, and the whole thing is checked from an ETS-cached plan lookup rather than a database read per request.

defmodule MyApp.Plans do
  use AuroraMeter.Plans

  plan :starter do
    price 2_900
    metered :ai_tokens, included: 500_000, unit_price: 2
    feature :api_access, true
  end

  plan :scale do
    price 9_900
    metered :ai_tokens, included: 5_000_000, unit_price: 1
    feature :api_access, true
  end
end

Then meter the real counts after the call returns:

def summarise(org, text) do
  with :ok <- AuroraMeter.check(org, :ai_tokens),
       {:ok, result} <- MyApp.AI.complete([%{role: "user", content: text}]) do
    AuroraMeter.track(org, :ai_tokens, result.input_tokens + result.output_tokens, durable: true)
    {:ok, result.text}
  end
end

durable: true matters here. Aurora Meter’s counters live in ETS and flush to Postgres on an interval, which is what makes the hot path fast, but it also means a VM crash can lose the increments since the last flush. Anything you are going to invoice should be marked durable so each increment also writes a raw event row synchronously. Use the buffered default for dashboards and rate limits, durable for money.

The prepaid shape, which is what tokens actually want

Here is the real problem. You need to reserve money before the call, because two concurrent jobs must not both spend the last dollar, and you can only charge the true amount after the call, because that is when you learn the token counts.

That is a hold and a settle, and with_credits/4 wraps both:

alias AuroraMeter.Credits
alias AuroraMeter.Credits.Money

def summarise(org, doc) do
  estimate = MyApp.AI.Pricing.estimate("gpt-4o", doc)

  Credits.with_credits(org, estimate, "doc:#{doc.id}", fn ->
    case MyApp.AI.complete([%{role: "user", content: doc.text}], model: "gpt-4o") do
      {:ok, result} ->
        cost = MyApp.AI.Pricing.cost("gpt-4o", result)
        {:ok, result.text, cost}

      {:error, reason} ->
        {:error, reason}
    end
  end)
end
# => {:ok, text} | {:error, :insufficient_credits} | {:error, reason}

Read what that does line by line, because every branch is one you would otherwise write by hand.

The estimate is reserved first, and Credits.available/1 reflects the hold immediately, so a second job started a millisecond later cannot spend the same balance. If the balance will not cover the estimate, the function never runs and you get {:error, :insufficient_credits} before you have paid a provider anything.

Returning {:ok, result, cost} settles the hold at the true cost. Returning an error tuple releases the hold and charges nothing, which is the correct behaviour when the provider returns a 500 and you were never billed. If your function raises, throws or exits, the hold is released before the error propagates, so a crashed worker does not leave money stranded.

Estimating is easy to get roughly right and impossible to get exactly right, which is fine. Take the input length you already have and assume the completion runs to max_tokens:

def estimate(model, doc) do
  approx_input = div(byte_size(doc.text), 4)
  cost(model, %{input_tokens: approx_input, output_tokens: 1_000, cached_tokens: 0})
end

Four bytes per token is a decent rule for English prose and wrong for code, JSON and non-Latin scripts. Bias the estimate high: an over-reservation is released on settlement, an under-reservation lets a customer overspend.

A settlement never fails for want of credit. If the document turned out to be five times longer than you estimated, the balance goes negative, which is the honest record of a debt you have already incurred with your provider. The next hold or debit is refused until a grant repairs it. That is a better outcome than a half-finished job and a charge you cannot explain.

Agent loops: one hold, many calls

A single completion is the easy case. An agent that plans, calls three tools and summarises makes an unknown number of calls of unknown size, and you cannot hold per step without the customer watching their balance flicker.

Hold a budget for the whole run and settle the sum:

def run_agent(org, task) do
  budget = Money.from_cents(50)

  Credits.with_credits(org, budget, "agent_run:#{task.id}", fn ->
    {result, spent} =
      Enum.reduce_while(1..max_steps(), {nil, 0}, fn _step, {_acc, spent} ->
        case step(task) do
          {:done, output, usage} -> {:halt, {output, spent + MyApp.AI.Pricing.cost("gpt-4o", usage)}}
          {:continue, usage} -> {:cont, {nil, spent + MyApp.AI.Pricing.cost("gpt-4o", usage)}}
        end
      end)

    {:ok, result, spent}
  end)
end

Now the budget is a real ceiling, visible to the customer before the run starts, and the settlement is the truth afterwards. Check spent against the budget inside the loop and stop early if the run is heading over, because that is a far better experience than a negative balance and a surprise.

Idempotency: the line that stops double billing

Every call above takes a reference string, and it is the most important argument in the API. It is the idempotency key.

A worker retries. A webhook is redelivered. A user double-clicks. Without a stable key each of those is a second charge for the same work. With one, the ledger recognises the reference and returns the original entry instead of writing a new one.

Key it on something that identifies the unit of work and does not change between attempts:

# good: stable across retries of the same job
Credits.with_credits(org, estimate, "oban_job:#{job.id}", fun)

# bad: a new key on every attempt, so every retry charges again
Credits.with_credits(org, estimate, "run:#{System.unique_integer()}", fun)

The same rule applies on the way in. When Stripe tells you a payment succeeded and you top up a balance, key the grant on the PaymentIntent id, because Stripe delivers each webhook at least once and retries failed deliveries for days:

Credits.grant(org, Money.from_cents(2_500), reference: "stripe:#{payment_intent_id}")

Without that reference a single $25 payment funds an account $50, then $75. Our post on prepaid credits in Aurora Meter goes deeper on the ledger itself, including promotional grants and expiry.

Show the customer what they are spending

Opaque AI billing is the fastest way to a chargeback. A customer who can see the number as it moves will accept it. A customer who sees it for the first time on an invoice will dispute it.

Three reads and two components, no JavaScript:

# mount/3
if connected?(socket), do: AuroraMeter.LiveView.subscribe(org)

<.credit_summary summary={AuroraMeter.Credits.summary(@org)} />
<.spend_chart points={AuroraMeter.Credits.spend_history(@org, days: 30)} />
<.usage_meter tenant={@org} feature={:ai_tokens} />

Credits.summary/1 returns the balance, the spend so far this period, the daily burn rate and the runway in days. Runway is the number worth putting in an email: telling a customer “at your current rate you have about four days of credit left” converts far better than a balance they have to reason about themselves.

The meters read the live ETS counters over PubSub, so a request metered on the server shows up in the browser within a second with no polling and no database read.

Getting the money out to Stripe

Metering and gating are the free core. Charging is where Aurora Meter Pro comes in, because that is the part that wants a provider integration rather than a library function: Stripe top-ups and auto-recharge so a balance refills itself at a threshold, metered usage reported to Stripe Billing Meters, billing periods aligned to the Stripe subscription instead of the UTC calendar month, and reconciliation between what you counted and what Stripe invoiced.

config :aurora_meter,
  repo: MyApp.Repo,
  pubsub: MyApp.PubSub,
  plans: MyApp.Plans,
  provider: AuroraMeter.Pro.Stripe

Pro is licensed per app from $99 a month with a 14 day free trial, claimed once per customer. There is a preview of what it does and the full breakdown on the pricing page.

The limits, stated plainly

Three things to know before you rely on any of this.

Buffered counters can lose usage. The ETS design is what makes track/3 cost a few microseconds instead of a database round trip, and the tradeoff is that a hard VM kill loses the increments since the last flush. Mark anything billable durable: true and accept the write.

Prices move without warning. Your pricing module is a snapshot of what a provider charged on the day you wrote it. Put a test around the margin calculation and a calendar reminder on the numbers.

Estimates are estimates. A customer can end a period slightly negative when a job blows past its reservation. Decide in advance whether you swallow that, invoice it, or block the next job, and put the choice in your terms rather than in a support ticket.

Put it together

Read the real token counts from the provider, price input and output separately in integer micro-dollars with a margin you chose on purpose, hold before the call and settle after, key every ledger write on something stable, and show the customer the number as it moves.

The free MIT core does the counting, the entitlement checks, the credit ledger and the LiveView components: it is on GitHub and the docs are here. Add it with {:aurora_meter, "~> 1.0"}, run one generated migration, and the code in this post works.

If you want the AI half already built as well, phx_ai ships the streaming LiveView chat and provider plumbing that the first section of this post assumes you have.