Testing Usage-Based Billing in Elixir: Quotas, Periods and Credits in ExUnit

A bug in a normal feature shows up as an error. A bug in usage-based billing shows up as money: a customer who was charged for 1,200 units when they used 1,000, a free account that quietly ran 40,000 AI generations, a quota that reset on the 2nd instead of the 1st. Nobody files a bug report for the second one, and the first one arrives as a chargeback.

That makes testing usage-based billing in Elixir worth doing deliberately, not as an afterthought to the feature tests. The good news is that the BEAM makes it pleasant: counters are in-memory, time can be frozen, and concurrency is cheap to simulate. This guide walks through a test suite that covers the eight ways metering usually breaks, using ExUnit and the test helpers that ship with Aurora Meter, the MIT-licensed metering and entitlements library for Phoenix. If you are new to the topic, start with the hub post on usage-based billing in Elixir and Phoenix and come back.

What actually goes wrong

Before writing tests, list the failures you are defending against. In our experience these are the ones that reach production:

1. Off by one at the cap. The 50th request on a 50-request plan is refused, or the 51st is allowed.
2. Races at the cap. Two requests arrive together when one unit is left, and both get it.
3. Period boundaries. Usage does not reset at the start of a new month, or resets at midnight in the wrong time zone.
4. Plan changes mid-period. An upgrade wipes the month’s usage (customer gets free units) or a downgrade instantly locks someone out with no warning.
5. Restarts. A deploy drops in-memory counters and everyone’s usage goes back to zero.
6. Clusters. Two nodes each count their own traffic and the totals overwrite each other instead of adding up.
7. Prepaid credit. A job of unknown cost spends money the customer does not have, or a failed job keeps the money.
8. The UI. The usage meter in the dashboard shows yesterday’s number while the API is refusing requests.

Each section below turns one or two of these into tests.

Setting up the test case

Aurora Meter ships AuroraMeter.Test, so the suite does not need to know how the runtime works internally. Wrap it in a case template next to your usual DataCase:

defmodule MyApp.MeterCase do
  use ExUnit.CaseTemplate

  using do
    quote do
      use MyApp.DataCase
      use AuroraMeter.Test
      alias AuroraMeter.Credits.Money
    end
  end
end

The library flushes counters to Postgres and broadcasts live values on timers. In tests you want to decide when that happens, so push the intervals out of reach and drive them by hand:

# config/test.exs
config :aurora_meter, flush_interval: 60_000, broadcast_interval: 60_000

flush!/0 and broadcast!/0 then run the flusher and the broadcaster on demand. Because the flusher is a background process, a test that flushes needs the shared sandbox mode that the standard Phoenix DataCase already uses for async: false tests.

Finally, a plans module for the tests to exercise. Use your real one if you can: the point is to test the rules you are actually going to charge people with.

defmodule MyApp.Plans do
  use AuroraMeter.Plans

  plan :free do
    price 0
    limit :generations, 50, :hard
    feature :exports, false
  end

  plan :pro do
    price 2_900
    limit :generations, 1_000, :hard
    feature :exports, true
    feature :seats, 5
  end

  plan :growth do
    price 9_900
    metered :generations, included: 1_000, unit_price: 3
    feature :exports, true
    counter :api_calls
  end
end

Isolation: one tenant per test

Counters live in a shared ETS table, so two tests that meter the same tenant will see each other’s usage. The simplest fix is also the one that keeps async: true working: give every test its own tenant key.

setup do
  org = unique_tenant()
  {:ok, org: org}
end

unique_tenant/1 returns something like "org_4182". The alternative, use AuroraMeter.Test, reset: true, clears the tables before each test but forces async: false. Prefer unique tenants and keep reset!/0 for the few tests that deliberately simulate a restart.

1. Test the plan rules at the exact boundary

Most billing bugs live within one unit of a limit, so test there rather than at round numbers far away from it:

defmodule MyApp.PlanRulesTest do
  use MyApp.MeterCase, async: true

  setup do
    {:ok, org: unique_tenant()}
  end

  test "free plan allows the 50th generation and refuses the 51st", %{org: org} do
    AuroraMeter.subscribe(org, :free)
    AuroraMeter.track(org, :generations, 49)

    assert {:ok, :ran} = AuroraMeter.with_quota(org, :generations, fn -> :ran end)
    assert AuroraMeter.usage(org, :generations) == 50

    assert {:error, :limit_exceeded} =
             AuroraMeter.with_quota(org, :generations, fn -> flunk("must not run") end)

    assert AuroraMeter.usage(org, :generations) == 50
  end

  test "growth plan bills past the allowance instead of refusing", %{org: org} do
    AuroraMeter.subscribe(org, :growth)
    AuroraMeter.track(org, :generations, 1_000)

    assert {:ok, :ran} = AuroraMeter.with_quota(org, :generations, fn -> :ran end)
    assert %{kind: :metered, overage: 1, unit_price: 3} = AuroraMeter.quota(org, :generations)
  end

  test "counters are measured but never refuse and never bill", %{org: org} do
    AuroraMeter.subscribe(org, :growth)
    AuroraMeter.track(org, :api_calls, 1_000_000)

    assert :ok = AuroraMeter.check(org, :api_calls)
    assert %{kind: :counter, overage: 0, limit: nil} = AuroraMeter.quota(org, :api_calls)
  end

  test "feature flags follow the plan", %{org: org} do
    AuroraMeter.subscribe(org, :free)
    refute AuroraMeter.entitled?(org, :exports)

    AuroraMeter.subscribe(org, :pro)
    assert AuroraMeter.entitled?(org, :exports)
    assert AuroraMeter.feature_value(org, :seats, 1) == 5
  end
end

Notice the flunk/1 inside the refused call. It proves the work did not run, which is the real promise of a hard cap. Asserting only on the return value would pass even if the expensive call had already happened. The overage maths gets its own deep dive in overage billing in Phoenix.

A raise should give the unit back

with_quota/3 reserves the unit before running your function. If the function fails, the customer should not pay for it:

test "a crash inside the gate releases the reservation", %{org: org} do
  AuroraMeter.subscribe(org, :free)

  assert_raise RuntimeError, fn ->
    AuroraMeter.with_quota(org, :generations, fn -> raise "model timed out" end)
  end

  assert AuroraMeter.usage(org, :generations) == 0
end

2. Race the last unit

A hard cap that holds for one request at a time and breaks under concurrency is not a hard cap. On a single node, with_quota reserves atomically in ETS, so you can prove it with a burst:

test "200 concurrent requests against 50 units admit exactly 50", %{org: org} do
  AuroraMeter.subscribe(org, :free)

  results =
    1..200
    |> Task.async_stream(fn _ -> AuroraMeter.with_quota(org, :generations, fn -> :ran end) end,
      max_concurrency: 50
    )
    |> Enum.map(fn {:ok, result} -> result end)

  assert Enum.count(results, &match?({:ok, :ran}, &1)) == 50
  assert Enum.count(results, &match?({:error, :limit_exceeded}, &1)) == 150
  assert AuroraMeter.usage(org, :generations) == 50
end

This is the test a hand-rolled “read the count, compare, then increment” implementation fails. Run it with a few different seeds and it will either pass every time or fail often; there is rarely an in between.

One honest caveat to encode in your expectations: across a cluster, each node enforces the cap against its own view, so a burst spread over several nodes can overshoot by what the other nodes admitted within one broadcast interval. That is a deliberate trade for keeping the hot path free of network round trips. If a contract says “never one unit over, across all nodes”, test against that requirement explicitly and design for it, rather than assuming it.

3. Freeze the clock at the period boundary

Waiting for the end of the month is not a test strategy. Everything in Aurora Meter reads time through a clock seam, and with_clock/2 pins it:

defmodule MyApp.PeriodTest do
  # the frozen clock is global to the node
  use MyApp.MeterCase, async: false

  test "usage starts fresh at the first instant of the new month" do
    org = unique_tenant()
    AuroraMeter.subscribe(org, :free)

    with_clock(~U[2026-01-31 23:59:59Z], fn ->
      AuroraMeter.track(org, :generations, 50)
      assert {:error, :limit_exceeded} = AuroraMeter.check(org, :generations)

      travel(~U[2026-02-01 00:00:00Z])

      assert AuroraMeter.period(org).start == ~U[2026-02-01 00:00:00Z]
      assert AuroraMeter.usage(org, :generations) == 0
      assert :ok = AuroraMeter.check(org, :generations)
    end)
  end
end

Periods are half-open UTC intervals, [start, end), so midnight on the 1st belongs to February and 23:59:59 on the 31st belongs to January. Test both sides of the line, because that is exactly where an off-by-one in a hand-written <= hides. A new period simply starts a new counter, so there is no reset job to forget to schedule, and nothing to test there beyond this.

Two things to know about frozen time. It is a single named process, so every module that uses it must be async: false. And because every row written inside the block gets the same timestamp, a test that writes several ledger rows and then pages through them should travel/2 a second between writes.

4. Plan changes keep the period’s usage

Upgrades are where customers are most sensitive to fairness, so pin the behaviour down:

test "upgrading mid-period keeps usage and reads it against the new limit", %{org: org} do
  AuroraMeter.subscribe(org, :free)
  AuroraMeter.track(org, :generations, 50)
  assert {:error, :limit_exceeded} = AuroraMeter.check(org, :generations)

  AuroraMeter.subscribe(org, :pro)

  assert AuroraMeter.usage(org, :generations) == 50
  assert AuroraMeter.remaining(org, :generations) == 950
  assert :ok = AuroraMeter.check(org, :generations)
end

test "downgrading over the new limit refuses new work but keeps the history", %{org: org} do
  AuroraMeter.subscribe(org, :pro)
  AuroraMeter.track(org, :generations, 300)

  AuroraMeter.subscribe(org, :free)

  assert AuroraMeter.usage(org, :generations) == 300
  assert {:error, :limit_exceeded} = AuroraMeter.check(org, :generations)
end

The second test documents a product decision as much as a code path. If your answer is “a downgrade takes effect at the end of the period instead”, that is a different design (a scheduled transition), and it deserves its own test.

5. Survive a restart

ETS is fast because it is memory, and memory goes away on a deploy. Aurora Meter flushes deltas to Postgres and seeds a cold counter from the last flushed value, so usage/2 is correct after a restart. Prove it by flushing, then throwing the in-memory state away:

defmodule MyApp.RestartTest do
  use MyApp.MeterCase, async: false

  test "flushed usage survives losing every in-memory counter" do
    org = unique_tenant()
    AuroraMeter.subscribe(org, :pro)
    AuroraMeter.track(org, :generations, 30)

    assert flush!() >= 1
    reset!()

    assert AuroraMeter.usage(org, :generations) == 30
  end

  test "unflushed usage is what a crash can lose" do
    org = unique_tenant()
    AuroraMeter.subscribe(org, :pro)
    AuroraMeter.track(org, :generations, 30)

    reset!()

    assert AuroraMeter.usage(org, :generations) == 0
  end
end

The second test is not a bug report. It documents the trade you made by buffering: anything since the last acknowledged flush is at risk if the VM dies hard (a clean shutdown flushes once more). Writing it down as a test keeps that fact visible to the next developer. For a feature where losing even a second of usage is unacceptable, mark it durable in config so every increment also writes an event row, and add a test that asserts the row exists.

reset!/0 clears global tables, which is why this module is async: false.

6. Simulate a second node on one machine

You do not need a real cluster in CI to test that counts add up across nodes. simulate_node/2 delivers the gossip another node would send, and simulate_flush/2 the total it would announce after flushing:

test "usage from another node adds to ours instead of replacing it", %{org: org} do
  AuroraMeter.subscribe(org, :pro)
  AuroraMeter.track(org, :generations, 2)

  simulate_node(:"web@10.0.0.2", [{org, :generations, 5}])
  assert AuroraMeter.usage(org, :generations) == 7

  simulate_flush(:"web@10.0.0.2", [{org, :generations, 40}])
  assert AuroraMeter.usage(org, :generations) == 42
end

The last assertion is the subtle one: after the other node flushes a database total of 40 (which already includes our earlier traffic as far as it has been flushed), our view is that total plus our own 2 unflushed units. The reasoning behind that arithmetic is in cluster usage metering without double counting.

7. Prepaid credits: hold, settle, refuse

Pay-as-you-go pricing moves the risk from “too many units” to “too much money”. The credit ledger holds an estimate before the work, settles the real cost after, and refuses when the balance cannot cover the hold. Each branch deserves a test:

alias AuroraMeter.Credits

test "settles the actual cost, not the estimate", %{org: org} do
  fund!(org, Money.from_cents(1_000))

  assert {:ok, "summary"} =
           Credits.with_credits(org, Money.from_cents(50), "job:1", fn ->
             {:ok, "summary", Money.from_cents(12)}
           end)

  assert credit_balance(org).available == Money.from_cents(988)
  assert credit_balance(org).held == 0
end

test "a failed job costs nothing", %{org: org} do
  fund!(org, Money.from_cents(1_000))

  assert {:error, :provider_down} =
           Credits.with_credits(org, Money.from_cents(50), "job:2", fn ->
             {:error, :provider_down}
           end)

  assert credit_balance(org).available == Money.from_cents(1_000)
end

test "refuses work the balance cannot cover", %{org: org} do
  fund!(org, Money.from_cents(10))

  assert {:error, :insufficient_credits} =
           Credits.with_credits(org, Money.from_cents(50), "job:3", fn ->
             flunk("must not run")
           end)
end

Amounts are integer micro-dollars, never floats, and Money.from_cents/1 keeps the tests readable. fund!/3 grants with a unique reference so tests never trip the ledger’s idempotency check by accident. If you want a test that does trip it on purpose (granting the same Stripe payment twice must not double the balance), call Credits.grant/3 directly with a fixed reference: twice and assert the balance once.

8. The live UI sees what the gate sees

A usage meter that lags the API is a support ticket waiting to happen. Subscribe the test process the same way a LiveView does, then drive the broadcaster:

test "a tracked increment reaches LiveView subscribers", %{org: org} do
  AuroraMeter.subscribe(org, :pro)
  AuroraMeter.LiveView.subscribe(org)

  AuroraMeter.track(org, :generations, 3)
  broadcast!()

  assert_receive {:aurora_meter, :usage, %{feature: :generations, value: 3}}
end

With the message contract covered here, your LiveView tests only need to check that the view renders whatever value arrives, which is ordinary Phoenix.LiveViewTest work. The guide to building a real-time usage dashboard shows the view side.

What not to test in ExUnit: Stripe

Unit tests should never call Stripe. The metering rules above are the part you own, and they are all plain function calls that run in milliseconds. The billing provider side (meter events reaching Stripe Billing Meters, invoices adding the right overage, subscription periods lining up with your counters) belongs in a separate, slower check against a Stripe sandbox, ideally with a test clock so you can advance a subscription past its period end and read the resulting invoice. Run it before a pricing change and before a release, not on every commit.

That integration is the part Aurora Meter Pro handles: Stripe Checkout and webhook sync, metered usage reported to Billing Meters, periods aligned to the Stripe subscription, quota alerts and reconciliation between your counters and what Stripe billed. Reconciliation matters for testing too, because it turns “we think the numbers match” into a report you can assert on.

A checklist for your own suite

• Every plan limit is tested one unit below, at, and one unit above the cap.
• Refused work is proven not to run (a flunk/1 inside the gate).
• A crash inside the gate does not consume quota or money.
• A concurrent burst admits exactly the remaining units on one node.
• Both sides of a period boundary are tested with a frozen clock.
• Upgrade and downgrade behaviour is written down as tests, not just in pricing copy.
• Flushed usage survives reset!/0, and the unflushed window is documented.
• Cross-node totals add up with simulate_node/2 and simulate_flush/2.
• Every credit path (settle, fail, refuse, duplicate grant) has an assertion on the balance.
• The live update message is asserted, so the dashboard and the gate agree.

Wrapping up

Testing usage-based billing in Elixir comes down to three moves: give every test its own tenant, take control of time and background work, and test at the exact edges where money changes hands. The BEAM makes each of those cheap, and a suite like this runs in a couple of seconds while covering the bugs that would otherwise surface as refunds.

The Aurora Meter core is free and MIT, with the test helpers used throughout this post included; the documentation covers installation and the testing guide in full. When you are ready for Stripe to start charging for the usage you are now confident in, Aurora Meter Pro is licensed per app from $99/mo, with a 14-day free trial (one trial per customer).