We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
From the blog
API Usage Tracking in Phoenix: Meter and Limit Usage per API Key
By Liam Killingback ·
API Usage Tracking in Phoenix: Meter and Limit Usage per API Key
If you sell an API, your customers do not have one API key. They have a production key, a staging key, one baked into a CI pipeline, one a contractor used last spring, and one that is about to end up in a public GitHub repository. You bill the customer, but the customer thinks in keys: “which of our keys used 40,000 requests yesterday?” is the first support ticket every API business gets.
This guide covers API usage tracking in Phoenix at the level of the individual key: authenticating keys properly, counting every request against both the key and the account that pays for it, putting an optional cap on a single key so a leaked one cannot drain the account’s quota, and showing the breakdown live in LiveView. The examples use Aurora Meter, the free MIT metering library for Phoenix, but the design holds whatever you count with.
If you are new to metering on the BEAM, Metering API Usage in Phoenix in Real Time with ETS explains why the counting happens in ETS rather than in Postgres. This post builds on it.
Two questions, two owners
Per-key tracking gets confusing when one counter tries to answer two different questions:
| Question | Who asks it | What it decides |
|---|---|---|
| How much has this account used this period? | Billing, plan limits | The invoice, and whether the next request is allowed |
| How much has this key used? | The customer, your support team | Debugging, attribution, revoking a misbehaving key |
The account is the billing tenant. The key is an attribution tenant. Keep them as two separate counters and both questions stay simple. Summing the keys to get the account total sounds tidy, but it breaks the day a key is deleted, and it turns every plan check into a query.
Step 1: Store API keys you can verify but never read back
Treat API keys like passwords: show the secret once, store a hash. Keep a short, non-secret prefix so customers can recognise a key in your dashboard and in their logs.
defmodule MyApp.Accounts.ApiKey do
use Ecto.Schema
import Ecto.Changeset
schema "api_keys" do
field :name, :string
field :prefix, :string
field :secret_hash, :binary
field :monthly_cap, :integer
field :revoked_at, :utc_datetime
belongs_to :org, MyApp.Accounts.Org
timestamps()
end
def create_changeset(key, attrs) do
key
|> cast(attrs, [:name, :monthly_cap])
|> validate_required([:name])
|> validate_number(:monthly_cap, greater_than: 0)
end
end
defmodule MyApp.Accounts.ApiKeys do
alias MyApp.Accounts.ApiKey
alias MyApp.Repo
@doc "Creates a key and returns the plaintext secret exactly once."
def create(org, attrs) do
secret = "sk_live_" <> Base.url_encode64(:crypto.strong_rand_bytes(24), padding: false)
changeset =
%ApiKey{org_id: org.id, prefix: String.slice(secret, 0, 14), secret_hash: hash(secret)}
|> ApiKey.create_changeset(attrs)
with {:ok, key} <- Repo.insert(changeset) do
{:ok, key, secret}
end
end
def verify(secret) when is_binary(secret) do
case Repo.get_by(ApiKey, secret_hash: hash(secret)) do
%ApiKey{revoked_at: nil} = key -> {:ok, Repo.preload(key, :org)}
_ -> :error
end
end
defp hash(secret), do: :crypto.hash(:sha256, secret)
end
A plain SHA-256 is the right hash here, unlike for user passwords. API secrets are long and random, so there is nothing for a slow hash like bcrypt to protect, and you look the key up on every request, so it must be fast. Add a unique index on secret_hash.
Step 2: Tell the meter what an org and a key are
Aurora Meter takes a tenant as the first argument to every call and turns it into a string key. By default that works for strings and integers. With two kinds of tenant you want both to be unambiguous, so an org with id 7 and a key with id 7 never share a counter. A small tenant module does that:
defmodule MyApp.MeterTenant do
@behaviour AuroraMeter.Tenant
@impl true
def to_key(%MyApp.Accounts.Org{id: id}), do: "org_#{id}"
def to_key(%MyApp.Accounts.ApiKey{id: id}), do: "key_#{id}"
def to_key(key) when is_binary(key), do: key
end
# config/config.exs
config :aurora_meter,
repo: MyApp.Repo,
pubsub: MyApp.PubSub,
plans: MyApp.Plans,
tenant: MyApp.MeterTenant,
history: true
history: true keeps a daily series per counter, which is what the per-key chart in step 6 reads.
Step 3: Plans for accounts, and a plan for keys
Accounts get the plans you sell. Keys get a plan too, which sounds odd until you notice what it buys you: a key’s counter becomes a declared feature, with a kind, instead of an anonymous number.
defmodule MyApp.Plans do
use AuroraMeter.Plans
# Plans customers buy
plan :free do
price 0
limit :api_requests, 10_000, :hard
end
plan :growth do
price 4_900
metered :api_requests, included: 250_000, unit_price: 1
end
# A plan every API key is subscribed to: measured, never blocked, never billed
plan :api_key do
price 0
counter :api_requests
end
end
counter is the right kind for a key. It is measured and it shows up in history and in quota/2, but it is never blocked and never billed: the account is what gets billed. Subscribe every new key to it when you create it:
with {:ok, key, secret} <- ApiKeys.create(org, attrs),
{:ok, _sub} <- AuroraMeter.subscribe(key, :api_key) do
{:ok, key, secret}
end
This matters more than it looks. A tenant with no subscription resolves to your :default_plan, which is usually :free. Without the subscription, a key would be checked against the free plan’s hard limit, a confusing bug that only appears once a key passes 10,000 requests.
Step 4: Authenticate and count in one plug
The plug reads the bearer token, verifies it, admits the request against the account’s plan, and attributes it to the key:
defmodule MyAppWeb.Plugs.ApiKeyAuth do
import Plug.Conn
alias MyApp.Accounts.ApiKeys
def init(opts), do: opts
def call(conn, _opts) do
with ["Bearer " <> secret] <- get_req_header(conn, "authorization"),
{:ok, key} <- ApiKeys.verify(secret),
:ok <- check_key_cap(key),
{:ok, :admitted} <- admit(key) do
AuroraMeter.track(key, :api_requests)
conn
|> assign(:api_key, key)
|> assign(:current_org, key.org)
|> put_quota_headers(key.org)
else
{:error, :key_cap_reached} -> halt_json(conn, 429, "This API key has reached its monthly cap.")
{:error, :limit_exceeded} -> halt_json(conn, 429, "Your plan's monthly request quota is used up.")
{:error, :not_entitled} -> halt_json(conn, 403, "Your plan does not include API access.")
_ -> halt_json(conn, 401, "Invalid or missing API key.")
end
end
# Reserve one unit of the account's quota. On a hard limit this is atomic on a node.
defp admit(key), do: AuroraMeter.with_quota(key.org, :api_requests, fn -> :admitted end)
defp check_key_cap(%{monthly_cap: nil}), do: :ok
defp check_key_cap(key) do
if AuroraMeter.usage(key, :api_requests) < key.monthly_cap,
do: :ok,
else: {:error, :key_cap_reached}
end
defp put_quota_headers(conn, org) do
q = AuroraMeter.quota(org, :api_requests)
conn
|> put_resp_header("x-quota-used", to_string(q.used))
|> put_resp_header("x-quota-remaining", to_string(q.remaining))
end
defp halt_json(conn, status, message) do
conn
|> put_resp_content_type("application/json")
|> send_resp(status, Jason.encode!(%{error: message}))
|> halt()
end
end
Wire it into an :api pipeline and every route behind it is authenticated, admitted and metered:
pipeline :api_authenticated do
plug :accepts, ["json"]
plug MyAppWeb.Plugs.ApiKeyAuth
end
A few decisions in that plug deserve explaining.
The order of checks. The key cap is checked before the account quota. A capped key that is over its cap should be refused without consuming a unit of the account’s quota, otherwise a leaked key could still burn through the account’s allowance one refused request at a time.
Admission at the edge. with_quota/3 reserves the unit and keeps it once the function returns. Calling it here, with a trivial function, means the unit is counted when the request is admitted, not when it finishes. For a typical API that is what you want: a request that reached your code was a request. If you only want to charge for requests that succeed, move the with_quota/3 call into the controller and wrap the real work in it, and the reservation is released if that work raises.
The key cap is advisory. It reads the key’s usage and compares, so two concurrent requests at the boundary can both pass. A per-key cap is a safety net against a runaway script or a leaked key, not a billing guarantee, and overshooting it by a handful of requests is fine. The account’s hard limit is the one that has to hold, and with_quota holds it atomically on a single node.
The quota headers. Returning x-quota-used and x-quota-remaining costs two ETS reads and saves your customers from guessing. For a metered plan, remaining comes back as :unlimited once there is no hard ceiling, so to_string/1 gives the header unlimited, which is honest.
Rate limits are a different job
Everything above is about a quota: how much in a billing period. It does nothing to stop a key sending 5,000 requests in one second. That is a rate limit, and you want both. Rate Limits vs Quotas in Phoenix covers where each belongs and how to run a per-key rate limit in front of the quota check.
Step 5: Revoking, rotating and deleting keys
Per-key tracking pays off the day a key leaks. The customer sees one key spiking in the dashboard, clicks revoke, and the spike stops. Revocation is just a timestamp, because verify/1 refuses any key with revoked_at set:
def revoke(%ApiKey{} = key) do
key
|> Ecto.Changeset.change(revoked_at: DateTime.utc_now(:second))
|> Repo.update()
end
Revoke rather than delete. A revoked key keeps its name, its prefix and its usage history, so the customer can still answer “what did that key do last month?” And because the account has its own counter, revoking or even deleting a key never changes what the account has used or what it will be billed. That is the payoff of the two-tenant design from the start of this post.
Rotation is create-then-revoke: issue a new key with the same name, let the customer deploy it, then revoke the old one. Both show in the dashboard during the overlap, which is exactly what someone mid-rotation wants to see.
Step 6: A live per-key breakdown in LiveView
Customers want two views: the account total against the plan, and a table of keys with their usage. Aurora Meter publishes usage changes over PubSub per tenant, so the page subscribes to the org and to each key:
defmodule MyAppWeb.ApiKeysLive do
use MyAppWeb, :live_view
alias MyApp.Accounts.ApiKeys
def mount(_params, _session, socket) do
org = socket.assigns.current_org
keys = ApiKeys.list_for_org(org)
if connected?(socket) do
AuroraMeter.LiveView.subscribe(org)
Enum.each(keys, &AuroraMeter.LiveView.subscribe/1)
end
{:ok, socket |> assign(:org, org) |> assign_keys(keys)}
end
def handle_info(_usage_changed, socket) do
{:noreply, assign_keys(socket, socket.assigns.keys)}
end
defp assign_keys(socket, keys) do
rows =
for key <- keys do
%{
key: key,
used: AuroraMeter.usage(key, :api_requests),
last_7_days: AuroraMeter.history(key, :api_requests, days: 7)
}
end
assign(socket, keys: keys, rows: Enum.sort_by(rows, & &1.used, :desc))
end
end
<.usage_meter tenant={@org} feature={:api_requests} />
<table>
<tr :for={row <- @rows}>
<td><%= row.key.name %> <code><%= row.key.prefix %>...</code></td>
<td><%= row.used %></td>
<td><%= row.key.monthly_cap || "No cap" %></td>
<td><%= if row.key.revoked_at, do: "Revoked", else: "Active" %></td>
</tr>
</table>
usage/2 reads the warm ETS counter, so re-rendering the table on every update costs no database query. history/3 does read Postgres; if a customer has dozens of keys, load the seven-day series once in mount/3 and only refresh the live totals in handle_info/2. Broadcasts are batched on Aurora Meter’s broadcast interval (one second by default), so a busy key does not flood the page. For the full dashboard treatment, Build a Real-Time Usage Dashboard with Phoenix LiveView goes further.
Billing stays on the account
Nothing about per-key tracking changes billing. The invoice is for the account, built from the account’s counter: hard-capped on :free, metered with an allowance on :growth. The key counters are counter features, and a counter is never billed, so there is no risk of charging a customer twice for the same request.
When you are ready for Stripe to do the charging, Aurora Meter Pro reports the account’s metered usage to Stripe Billing Meters, aligns billing periods to the Stripe subscription and keeps plans in sync with Checkout and webhooks. The key counters stay entirely in your app.
Honest limits of this design
- Clusters loosen the hard limit slightly. On several nodes, each node enforces the account’s hard limit against its own view, so a burst can overshoot by what other nodes admitted within one broadcast interval. For API quotas measured in thousands, that is usually noise. The Aurora Meter docs have the arithmetic.
- Counters are buffered. Usage lives in ETS and flushes to Postgres on an interval. A hard VM crash loses the unflushed window. For a request counter that is acceptable; for something you bill per unit at a high price, mark the feature durable so each increment also writes an event row.
- One counter per key per period. Thousands of keys is fine. Minting a key per request (some “ephemeral token” designs do this) would create a counter per request, so attribute those to their parent key instead.
-
Per-key caps are advisory, as discussed above. If a customer needs a key-level limit that must hold exactly, give that key its own plan with a
limitand admit against the key withwith_quota/3as well.
Or skip the plumbing
The free core is everything this post used: ETS counters, the plan DSL, with_quota, history and the LiveView components. It is MIT, on GitHub, and the core docs take about ten minutes to get running. If you would rather start from a working API product, the Aurora API Starter template ships API keys, metering and a usage dashboard on top of it.
When usage needs to become Stripe invoices, Aurora Meter Pro is licensed per app from $99 a month, with a 14-day free trial (one trial per customer).
Summary
API usage tracking in Phoenix gets simple once you separate the two questions:
- Store API keys as a SHA-256 hash with a visible prefix, and show the secret once.
-
Give orgs and keys distinct tenant keys (
org_7,key_7) with a small tenant module. -
Bill and limit on the account; attribute on the key with a
counterfeature, and subscribe every key to that plan. - Authenticate, check the key cap, admit against the account quota, and track the key, all in one plug.
- Revoke keys rather than deleting them, so their history survives and the account total never moves.
- Show the account meter and a live per-key table in LiveView, reading warm ETS counters.
Your customers get the answer to “which key did that?” in one glance, and a leaked key becomes a two-click fix instead of a surprise invoice.