Why your ROAS and your incremental ROI don't match

The network reports a 7.4x ROAS on the campaign. Your own measurement comes back at 2.0 incremental. The first instinct is that somebody made a mistake.

Usually nobody did. Both numbers can be calculated correctly and still land nowhere near each other, because they start from different assumptions about what counts. That gap is where a lot of budget decisions get made, so it’s worth knowing the factors that go into those numbers.

They're answering different questions

Ad-attributed sales and ROAS tell you what happened, not whether marketing caused it. Attributed ROAS is descriptive, it quantifies how many exposed consumers purchased within a time window. It doesn’t tell if those sales still would have happened without advertising. 

While that dimension of causal inference vs. descriptive is the primary distinction between the two, there are other contributing factors which can drive those numbers differences. 

Does the attribution window matter?

Yes. Seven, fourteen and thirty days produce different attributed sales on the identical campaign, and the industry still hasn't standardized. Longer windows can sweep in more purchases that could have happened anyway, which pushes attributed ROAS up without touching the incremental number. Conversely, for upper funnel campaigns that often have delayed effects on behavior, a shortened window is missing sales that campaign likely influenced.

Even with a clean control group, Incrementality tests can fall into the same issues. Lift tests still rely on a time window. They compare aggregate outcomes across groups over a fixed period, so window length is still a lever that can drive differences. 

What about impressions nobody clicked?

Depends entirely on the network, and this one cuts both ways.

When view-through attribution is on, a shopper who scrolled past a banner and bought three days later gets counted. That inflates attributed sales considerably, particularly for ad-units that generate lots of impressions but few clicks.

When attribution is click-only, the opposite happens. Real influence from impressions goes uncounted, so attributed ROAS can understate what the media did. 

Any experiment framework particularly those that rely on synthetically creating a control group after the fact needs to keep this in mind as well to avoid contamination. This can be further complicated in user-level test design, where questions of cookies, device graphics, households or individuals, etc. can shape group design. 

Which sales are being counted?

The network can track sales in its own store. But your brand sells through several other retailers and in a DTC channel. 

A shopper who sees a sponsored ad and buys at a different retailer will register as a non-converter. From where the network sits, nothing happened. But a sale did happen on your P&L. Geo-based incrementality reading total sales in the market may catch it.

That breadth can cut against you too. If your campaign at Retail A moves a purchase that would have happened at Retailer B, Retail A counts a sale and your company-level read correctly shows nothing new was created. Attribution can't see substitution across retailers.

Who saw the ad?

This is the biggest single driver as it directly pertains to the definition of incrementality and yet it often gets discussed the least.

Onsite sponsored search reaches shoppers who are already searching your category on the retailer's site. It’s the highest-intent audience in commercial media. Most of them already have some baseline propensity to buy from the category or perhaps even your brand. Because attributed ROAS only describes that they converted not if they would have converted anyways, there can often be a very significant difference between ROAS and iROAS.  

How you build the control group should account for this. An onsite holdout must draw from people already shopping the site, so both groups are full of buyers who are potentially going to buy.

What else was running?

ROAS doesn’t see your national TV, ad or the price cut that landed mid-flight. None of it appears in the attribution logic. That tailwind or headwind can drive a massive difference in performance particularly if it is unevenly distributed across consumers. 

Experimental design must account for this as well. Draft from user or geography where through randomization and/or design equality reflect potential external exposure.

Reading the gap

Some difference between ROAS and incrementality should be expected and reflects the essential differences between the two. Before you argue why the difference is large or small, work through the list: window length, view-through settings, sales universe, audience intent, concurrent activity.

Then use both numbers rather than picking one. Attribution tells you whether media reached buyers, which is real information for keyword, placement, and format decisions inside that environment. Incremental ROI tells you whether the ads produced sales the company wouldn't otherwise have made.