The outbox, and the item nobody can explain

Telling Stripe what a customer used is a network call, and network calls have three outcomes, not two. It worked, it failed, or you do not know. The third one is where money goes missing in both directions, and it is the reason Pro stages every export as a row in your own database before it talks to anybody.

This guide is the outbox: the states, the lease, the key that makes a retry safe, and what a person does with an item nobody can explain. It needs Aurora Meter Pro.

#Why a row first

The old shape was a worker that read a counter, called Stripe, and wrote down what it had done. If the process died between the call and the write, the next run had no idea whether Stripe had heard.

The outbox inverts it. The intent to export is committed to your database in the same transaction as the fact it exports. Then a separate worker delivers it and records the outcome against that row. Nothing is ever in flight without a row that says so.

config :aurora_meter, events_outbox: MyApp.Outbox

#The states

An item is in exactly one of seven:

State	Meaning
pending	staged, waiting for a worker
leased	a worker has claimed it and is working on it
accepted	the provider took it
confirmed	the reconciler has since seen the provider’s own evidence of it
uncertain	nobody knows whether it landed
rejected	the provider refused it, or a person decided it never landed
quarantined	it cannot be sent as it stands, and sending it would be wrong

Two of those deserve emphasis. rejected is terminal and visible, and nothing automatic ever leaves it. And running out of attempts does not produce rejected: exhausted work becomes uncertain, because “I tried twenty times and stopped” is not evidence that the provider never heard.

#Identity, and the key that makes retries safe

Every item carries a seven part identity: the tenant, the subject kind (usage_window, event or correction), the subject reference, the exporter, the destination account, the mode (test or live) and the mapping version. That tuple is unique in the database, and the payload is immutable once written.

The identifier sent to the provider is a SHA256 of four of those parts. It is a pure function of the identity, with no nonce and no timestamp, which is the whole point:

A nonce here would not be unnecessary. It would make every retry a second charge. The idempotency key must be the same value on every attempt, which means it cannot be generated at send time.

For a usage window the subject reference carries both bounds, for example usage:api_calls:2026-09-01T00:00:00Z:0:5. Restaging a window nobody acknowledged produces the same identity and collapses onto the existing row. A later window, after an acknowledgement, is a different identity and a different key, which is correct: it is different work.

#The lease is two mechanisms, and only one of them is a clock

This is worth getting straight, because “the lease expired so my write lost” is a common and wrong mental model.

Who gets an item is decided by SELECT ... FOR UPDATE SKIP LOCKED inside the claiming transaction. No timestamp is involved.

Whose write wins is decided by lease_token, an integer counter incremented on every claim. Every transition out of leased carries the token it was given. A worker that was re elected out from under writes zero rows and is told {:error, :stale_lease}. No timestamp is involved there either, so a clock stepping backwards cannot invert the decision.

lease_expires_at is the one duration that is compared, and it decides only one thing: when an abandoned item becomes claimable again. It is liveness, not correctness.

Zero rows updated is a first class outcome here. It is never an exception and it is never retried.

The knobs, with their defaults:

config :aurora_meter_pro,
  outbox_lease_seconds: 120,
  outbox_max_attempts: 20,
  outbox_batch_size: 100,
  outbox_backoff_base_seconds: 5,
  outbox_backoff_cap_seconds: 900,
  uncertainty_horizon_seconds: 82_800

That horizon is 23 hours, and it is not arbitrary: it sits inside Stripe’s minimum 24 hour idempotency window, so a retry within it lands on Stripe’s own record of the original request rather than being treated as new work.

#Quarantine

An item is quarantined when it cannot be sent as it stands: the feature is not mapped to a meter, there is no customer, the account is not confirmed, the mode does not match, the period belongs to the buffered era, the timestamp is outside the provider’s window. The reason is on the row.

Quarantine is not a failure to be retried. It is a configuration problem with a name. Fix the cause, then requeue the item deliberately, which puts it back to pending. A requeue whose mapping version has changed in the meantime is refused with {:error, :mapping_changed}, because the item was built for a mapping that no longer exists.

#An uncertain item, and the person who resolves it

uncertain means Pro cannot tell whether the request landed. Three things produce it: an exporter that answered uncertain, a retry past the attempt cap, or a retry past the horizon.

Nothing in the software moves it. There is no automatic retry, no sweeper, no eventual timeout that decides for you. That is deliberate, and the asymmetry is why:

A wrong acknowledgement under bills by one window. A wrong abandonment double bills, because the window restages under a new identity and is sent again. When you are not sure, leave the item uncertain.

The work is: find the items, look each one up in the provider’s own records, and record what you found.

AuroraMeter.Pro.Recovery.list_uncertain(older_than: 3_600, limit: 50)
AuroraMeter.Pro.Recovery.inspect_item(item_id)

Then one of two calls, each of which requires you to state what you expect to be true, so a stale screen cannot act on a row that has since moved:

AuroraMeter.Pro.Recovery.acknowledge_item(item_id,
  expected: %{state: :uncertain, lease_token: 3, subject_ref: "...", provider_ref: "..."},
  reason: "verified in the Stripe dashboard",
  evidence: "Stripe request log req_TEST at 2026-09-15T09:12Z shows a 200 for this identifier"
)

Acknowledging moves the item to accepted, not to confirmed. Only the reconciler, holding the provider’s own evidence, may say confirmed.

AuroraMeter.Pro.Recovery.abandon_item(item_id,
  expected: %{state: :uncertain, lease_token: 3, subject_ref: "..."},
  reason: "no meter event for this identifier in the request logs",
  evidence: "searched 2026-09-15 09:00Z to 10:00Z, 0 results"
)

Abandoning moves it to rejected and clears the staged link, so the window restages under a new identity. That restaging is a send. Be sure.

Both calls need reason and evidence, both write a recovery action row whether they applied or refused, and neither has a force: option. The expected state mismatch answer is {:error, {:expected_state_mismatch, current}} and tells you what the row says now.

The same work from a terminal:

mix aurora_meter_pro.recovery list --state uncertain --older-than 24h
mix aurora_meter_pro.recovery inspect <id>
mix aurora_meter_pro.recovery acknowledge <id> --provider-ref ... --reason ... --evidence ... --yes

Read only by default. Without --yes it prints the row, the change it would make and the expected state map, and exits without writing.

#Reconciliation

Two different questions, answered by two different functions.

The local check needs no network:

AuroraMeter.Pro.Reconcile.run()
# %{local_drift: [...], items_open: 3}

It flags any ledger position that claims to have reported more than the counter it reported, which is a physically impossible state. It mutates nothing. Be clear about its blind spots: it can see over reporting; it cannot see under reporting, an asynchronous rejection by the provider, or anything about an invoice.

Provider findings come from the reconciler comparing your committed facts and export receipts against Stripe’s meter event summaries, error reports and invoices. Each difference becomes a reconciliation item:

AuroraMeter.Pro.Reconcile.list_items(state: "open", limit: 100)

An item carries a kind (there are ten, from summary_shortfall to unmapped_price to reversal_without_grant), the tenant, the feature, the period, what was expected, what was observed, and detail.

AuroraMeter.Pro.Reconcile.resolve_item(id, "credited manually, see invoice in_123",
  actor: "liam"
)

Read that carefully: resolving changes nothing else. No counter moves, no outbox item moves, no credit is granted. It records that a person looked at the difference and what they did about it. The fixing is your job; the item is the record that it happened.

A blank or whitespace resolution is {:error, :invalid_resolution}, because an empty explanation is worse than an open item.

#Cutting a feature over from counting to recording

This is the one moment when both reporters could claim the same period, and that is a double bill. The watermark makes the split exclusive:

• the buffered reporter stages a window only while its period starts before the watermark
• the outbox accepts an event only while its period starts on or after it, and quarantines anything earlier with a reason of period_buffered

The procedure, in order, and the order is the point:

mix aurora_meter_pro.cutover --feature api_calls --at 2026-10-01T00:00:00Z --tenant org_123

Keep the reporter running. Watch until the buffered side has drained:

AuroraMeter.Pro.Cutover.status(:api_calls)
# %{watermark: ~U[2026-10-01 00:00:00Z], buffered_periods_open: 0,
#   event_items: 412, drained?: true}

Only then deploy the config that changes where the quantity comes from:

config :aurora_meter, feature_sources: %{api_calls: :events}

The Mix task never writes that line. Moving the watermark and changing the source are two decisions, and collapsing them into one command would mean a deploy could accidentally do both.

A feature that already has a scheduled cutover answers {:error, :already_scheduled}. Moving the date goes through AuroraMeter.Pro.Recovery.reschedule_cutover/2, with the same expected state and evidence discipline as everything else in Recovery, rather than through a second schedule call.

#What to watch

• outbox backlog age, from the [:aurora_meter, :pro, :outbox, :gauge] event. The gauge raises rather than emitting a zero if its query fails, because no sample is better than a wrong one.
• the count of uncertain items. Any number above zero is somebody’s afternoon.
• quarantine growth. A steady climb is a mapping that was never finished.
• open reconciliation items older than a day.

#What to read next

• Report usage to Stripe is the layer above this one.
• Durable events and corrections is what the outbox exports.