Case study · Attribution & incrementality

Paid marketing looked unprofitable until we counted store sales.

Web analytics said 0.9 ROAS. Store stitching said 4.0. Channel holdouts said 2.4. The last number is the one budgets run on.

When2025–2026 · in production
RoleLed a 3-person build · architecture & pipelines mine
DataBehavioral sessions · tag events · omni orders · ERP store transactions
MethodsIdentity stitching · tiered last-touch · channel holdouts · ERP calibration

The problem

2,300 stores. Most journeys start online; most sales close in a store. Our channel reporting was web-only last-click, so it graded marketing on the fifth of revenue it could see and filed the rest under Direct. By that math the whole paid portfolio lost money. Nobody believed it. Nobody could prove it wrong, either.

The build

Three of us: a tagging expert, a developer, and me on architecture and pipelines.

What it showed

Three verdicts on the same week of spend:

One week of paid spend, graded three ways
ROAS by channel · web-only last-click vs. store sales stitched on vs. holdout-calibrated incremental · figures perturbed
break-even 0.9 4.0 2.4 2.7 10.5 4.0 0.4 2.7 1.9 0.5 2.2 1.7 0.9 2.1 1.4 3.0 4.4 1.5 Blended paid Brand search Local inventory Non-brand Shopping Affiliates web-only + store sales, attributed incremental (holdout-calibrated)
The haircuts are the story. Brand search kept under half its attributed credit; people who search your name were mostly coming anyway. Affiliates kept a third; coupon sites were taking credit for orders already in motion. Local inventory ads went from the worst line on the web report to solidly profitable, because most of what they drive is a store sale the old report never saw.
The channel that looked worst was underpriced. The channel that looked best was overpaid. The report was the problem.

What changed

Budgets reallocate on the incremental numbers now. Every other dashboard's money rows must foot to this pipeline to the dollar, because executives add reports together and a mismatch kills trust in both. Ecommerce credit reads next morning; store credit waits out the match window instead of reporting early and wrong.

Two bugs worth mentioning. A model in the stack labeled "Shapley" was not Shapley; its coalition math was broken, so I rebuilt it, then relabeled it honestly instead of shipping the name. And a converter rule requiring a payment pageview was dropping every order paid through a wallet redirect. Fixing it lifted order capture from about 70% to 95%.

Technical notes, for the people who will ask

Identity. Visitor IDs extracted from the behavioral stream, linked across devices, with hashed-email and analytics-ID edges added by co-occurrence. Guards: identities shared across several distinct emails are dropped; bot networks filtered. Orders come from three sources unioned, because each covers the others' failure modes.

Credit rules. 30-day lookback at or before the order timestamp. Tiered last-touch: paid, then earned, then fallback; last touch in the highest tier wins. Converters with no attributable touch redistribute proportionally instead of inflating Direct, which browser privacy already inflates.

Calibration. Stitched proportions scale to ERP totals per order channel. A year-over-year stitch-improvement correction keeps better stitching from masquerading as growth. Provisional days are flagged, not passed off as final.

Incrementality. Rotating matched-market holdouts, one channel at a time: comparable markets go dark for several weeks, matches keep spending, and the sales gap yields the per-channel factor applied to attributed returns. Sanity checks: a second attribution model agrees on awareness share to within tenths of a point, and a third-party affiliate network agrees on about 70% of contested converters.

The other case studies: the organic decline, the mart, and the dashboard suite. Or back to all work.

ROAS figures, stitch rates, and counts are perturbed; incrementality factors reflect industry-typical ranges by channel; channel names are generalized and vendor names removed. The architecture, the corrections, and the decisions are real.