A ledger under concurrent transfers

Every strategy is given the same thousand transfers. Read-modify-write commits them all, quickly, and leaves the ledger wrong: two transfers that read the same balance before either writes lose one of the updates. In the recorded run 64 accounts ended up disagreeing with their own entries. Every other strategy keeps the ledger right; what differs is the price.

Locks taken payer first deadlock. In the recorded run PostgreSQL broke 64 lock cycles, each after a second of waiting, while the transfers behind them queued: when the run stopped starting new ones, at twenty seconds, only 121 of the thousand had begun. The same locks taken in id order, or a guarded update, commit all of them in about half a second. Optimistic versions were retried 538 times and serializable transactions 907: correct, slower, and a few transfers ran out of attempts.

The invariants are SQL run on the database after each run, not counters kept by the code under test: the balances add up to what they started at, none is below zero, every transfer’s two entries sum to zero, and each cached balance equals its opening balance plus its entries. The third is also a deferred constraint trigger: a transaction that leaves a transfer unbalanced cannot commit, whatever the strategy got wrong.

Aborts are counted by their SQLState, never swallowed: 40P01 when PostgreSQL breaks a deadlock, 40001 when a serializable transaction cannot be serialized, and a version mismatch for the optimistic strategy. Each is retried after a short random wait, up to ten attempts.

After a live run the page draws the run’s trace: opening the pool, the reset, the sixteen workers with each worker’s transfers one after another, and the SQL of the checks. The spans come from Micrometer observations, recorded by the OpenTelemetry SDK and kept in memory on the server for the last eight runs; nothing is exported anywhere.

The first trace caught a flaw in this experiment. The pool opened its sixteen connections one after another while the workers waited for theirs: in one run the last worker began 983 ms after the run did, and for the first 300 ms only three were moving money, so “sixteen at a time” held only towards the end. The pool is now filled before the clock starts. With every worker starting together the strategies met more contention: in the runs repeated then, optimistic versions conflicted 574 to 597 times where they had 468, and locks taken payer first finished 123 to 147 transfers where they had 333.

The database’s own log caught another. Every attempt began by setting its transaction’s isolation level again, a round trip of its own: a thousand of them in a run that retried nothing. The level is set once for each connection now, sixteen times in a run, and the recorded runs were made again.

At Worldline I worked on the critical transaction path of card payments, over Oracle and PostgreSQL. The query under Reconciliation here is a safety net that systems moving money keep: a cached number checked against the entries it should be the sum of, so that a bug like the first strategy’s is found by a job, not by a customer.