# When Attribution and Incrementality Disagree, Which Decision Changes?

A high reported ROAS and a modest experimental lift can both be valid. Separate their questions, test the commercial threshold and decide what the evidence supports changing.

Source: https://dtcup.com/blogs/insights/attribution-incrementality-disagreement-business-decision/

DTCUP / FIELD NOTES / ACQUIRE

# When Attribution and Incrementality Disagree, Which Decision Changes?

A high reported ROAS and a modest experimental lift can both be valid. Separate their questions, test the commercial threshold and decide what the evidence supports changing.

Published Oct 2, 202625 min read

 ![Jack Hu](https://dtcup.com/wp-content/plugins/dtcup-knowledge/assets/jack-hu-small.jpg)**Jack Hu**7 years of experience building and operating brand websites, specializing in website development, paid advertising and SEO. Built a brand website from launch to US$10 million in annual sales.



  On this page  ## The Short Answer

First check whether the reports describe the same population, period and decision. Attribution assigns credit; a credible experiment estimates the effect of a particular change. Compare that effect and its uncertainty with contribution economics, then choose a bounded action instead of forcing both reports to agree.

## Key takeaways

- Credit and causal effect answer different questions.
- A valid study can still leave the commercial decision uncertain.
- Positive incremental revenue does not automatically cover cost.
- An inconclusive interval is different from evidence of a poor return.
- The value of current spend does not establish the return on extra spend.
- Do not turn one experimental ratio into a universal correction factor.

![Sage glass panels with traced paths, a brass prism and a mechanical balance sit behind a horizontal brass reference rail.](https://dtcup.com/wp-content/uploads/2026/10/dtcup-a08-operating-guides-20261002-1024x576.webp)The advertising dashboard reports a return of six dollars for every dollar spent. A controlled study suggests that the same activity generated only three additional dollars per dollar of additional cost. The immediate reaction is often to ask which report is wrong.

That question can lead the team in the wrong direction. One report may be allocating credit to observed conversions while the other estimates how outcomes changed under a particular advertising intervention. Both can be correctly calculated and still support different decisions.

I would begin by writing the decision that is actually on the table. Are we deciding whether to keep the tested activity, whether to increase its budget, which creative needs attention or whether the measurement pipeline is broken? Those questions require different evidence. A conflict between two headline ratios should not automatically trigger a budget cut or a change to the attribution model.

This article is a guide to interpreting the disagreement after evidence arrives. It is not another experiment setup checklist. We will use a fictional brand, Linden Field, and explicitly invented study summaries to show how scope, uncertainty, contribution and operational limits change the next action. None of the numbers is a measured campaign result or a recommended spending level.

## Translate each number into the question it answers

An attributed conversion is a conversion receiving credit under a measurement rule. That rule can use a click, a view, a sequence of recorded interactions, a modeled assignment or another supported signal. The resulting number depends on the conversion definition, evidence coverage, attribution window and credit logic.

An incrementality estimate asks how the defined outcome changed because of the tested difference in activity, relative to a credible comparison. It depends on the intervention, assignment or identification method, observation window, outcome coverage and assumptions. It does not necessarily identify which individual orders were caused by advertising.

The same customer can illustrate why these questions differ. A loyal buyer may click a useful reminder and purchase. The reminder receives credit under a click-based rule. Whether that buyer would have purchased without the reminder is a different question that cannot be answered simply by inspecting the click.

The reverse is possible too. Advertising can influence a person who later returns through an untracked route or another channel. An attributed report may give little or no credit to the original activity even when a well-designed comparison detects an effect on the outcome.

Neither example proves that all attribution overstates or understates value. The direction and size depend on the activity and measurement conditions. A general belief that one channel is always overcredited is not a substitute for investigating the actual evidence.

Write a plain-language caption for every number used in the decision. For example: revenue credited to this campaign under this window; estimated additional qualifying revenue from enabling this activity in these markets; or total store net revenue in this period. If two captions describe different objects, their difference is not automatically a reconciliation error.

The purpose is not to make the reports agree. It is to stop a number designed for one question from answering a different question without justification. That discipline preserves useful operating signals while making causal decisions more defensible.

[![Attributed revenue allocates credit under a measurement rule, incremental revenue estimates the effect of a defined intervention, and commercial contribution subtracts the relevant costs for a decision.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-questions.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-questions.png)Credit, effect and commercial value are connected, but they are not interchangeable.## Audit comparability before calling the difference a contradiction

Start with the intervention. A study of campaign presence versus absence estimates the effect of that contrast under the surrounding marketing arrangement. A study of a budget increase estimates the effect of that particular increase. Neither is automatically an answer about the whole channel at every future spending level.

Then align the population. A platform report may include all markets while the experiment covers a subset. A returning-customer audience test does not establish the effect for first-time prospects. A mobile-only implementation may not support a conclusion about every device or order path.

Align the outcome and money definition. Gross order value, net merchandise revenue and contribution are different measures. Taxes, shipping charges, cancellations, returns and currency conversion can create legitimate differences. Use the same definitions where comparison is intended and preserve the distinction where it is not.

Align the clocks. An attribution report may credit conversions to an interaction date while an experiment counts outcomes during a defined observation period. Delayed orders and later refunds can change the apparent relationship. A cutoff that is convenient for a meeting is not necessarily the planned mature observation window.

Check the cost denominator as carefully as the revenue numerator. In an increase-versus-baseline study, incremental cost is the difference between conditions, not automatically the treatment group’s entire advertising bill. In a presence-versus-absence comparison, the relevant difference may be closer to the tested activity’s full cost, but it still needs verification.

Inspect implementation before interpreting an uncomfortable result. Did the intended activity differ between conditions? Did another campaign replace the withheld delivery? Did tracking work comparably? Did a market-specific outage or stock shortage change the comparison? These are questions about validity, not excuses to reject a result because it is disappointing.

Label the outcome of this audit. The reports may be not comparable yet, comparable but answering different questions, or genuinely inconsistent with the stated data definitions. Each status needs a different response. Repairing an event join is not the same action as revising a budget belief.

For detailed source reconciliation, use [the guide to conflicting attribution reports](https://dtcup.com/blogs/insights/reconcile-attribution-report-differences/). Here the goal is to establish whether there is a real decision disagreement before deciding how much weight to give each form of evidence.

## Read one worked disagreement without selecting a favorite ratio

Linden Field runs a hypothetical presence-versus-absence study for a defined activity. The example assumes that implementation and analysis checks are satisfactory, the measured outcome is mature net merchandise revenue and the incremental advertising cost is fixed at thirty thousand dollars.

For the same relevant treated scope, the platform credits one hundred eighty thousand dollars of revenue to the activity. Dividing by thirty thousand gives an attributed ROAS of six. This is an allocation under the chosen rule, not a statement that all credited revenue disappears without the advertising.

The fictional study summary estimates ninety thousand dollars of additional revenue, with a stated ninety-five percent confidence interval from fifteen thousand to one hundred sixty-five thousand. These interval values are supplied for teaching; they are not calculated from hidden data or offered as a real statistical result.

With the assumed fixed cost, the point estimate of incremental revenue per advertising dollar is three. The corresponding illustrative interval is one-half to five and one-half. The width is central to the interpretation: the evidence does not support treating three as an exact multiplier.

The attributed and incremental figures differ because they describe credit and estimated effect. Their difference is not a set of named fake orders. You cannot take the ninety thousand dollar gap, identify particular customers in the order ledger and declare that those exact purchases would have occurred anyway.

Nor does the study make the platform’s interaction data useless. It may still help identify broken links, creative fatigue, landing-page differences and changes in the mix of recorded traffic. The lesson is to use those signals within their limits rather than turning every credited dollar into a causal claim.

The next decision still needs economics. Ninety thousand dollars of additional revenue can be attractive or unattractive depending on the contribution generated by those sales and the costs required to create them. A positive revenue effect alone does not establish that the activity covers its cost.

Keep the complete example together in the review. Showing only six supports an overconfident expansion story. Showing only three hides uncertainty. Showing only a wide interval without the business threshold can leave the team confused about why the uncertainty matters.

[![A fictional activity has thirty thousand dollars of incremental cost, one hundred eighty thousand attributed revenue and ninety thousand estimated incremental revenue with a fifteen to one hundred sixty-five thousand interval.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-example.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-example.png)The example’s interval is supplied to illustrate interpretation, not estimated from an actual campaign.## Put the commercial threshold inside the uncertainty range

Assume Linden earns a constant forty percent contribution on the measured incremental revenue before the tested advertising cost. This is a simplifying assumption. A real calculation should use the relevant sales mix and costs, and may need direct contribution outcomes instead of one margin percentage.

At the ninety thousand dollar point estimate, contribution before advertising is thirty-six thousand dollars. Subtracting the thirty thousand dollar incremental cost leaves six thousand dollars. This is contribution after the specified cost, not a claim about company net profit.

Under the same fixed-margin and fixed-cost assumptions, the supplied revenue interval maps to a contribution-after-advertising interval from negative twenty-four thousand to positive thirty-six thousand dollars. That range includes commercially unattractive and attractive outcomes.

The break-even incremental revenue is seventy-five thousand dollars, because forty percent of that amount covers thirty thousand dollars of advertising cost. The corresponding break-even incremental ROAS is two and one-half. The important boundary is therefore not only zero lift; it is the return required for the stated business decision.

A study can support a positive revenue effect while leaving profitability unresolved. It can also find a positive effect too small to cover the relevant cost. Treating every statistically positive result as permission to scale skips the economic question.

Consider a second invented study summary with forty-five thousand dollars of additional revenue and an interval from thirty-five to fifty-five thousand, using the same thirty thousand cost and forty percent contribution assumption. The mapped contribution after cost ranges from negative sixteen thousand to negative eight thousand. The effect is positive, yet this illustrative range lies below the commercial threshold.

A third invented summary estimates one hundred twenty thousand, with an interval from ninety-five to one hundred forty-five thousand. Under the same assumptions, contribution after cost ranges from eight thousand to twenty-eight thousand. That evidence is more supportive of the tested activity at the tested scope, though it does not prove unlimited expansion would earn the same return.

These examples show why significance labels alone are an incomplete decision rule. Put the range on a business scale, identify the threshold and state the assumptions that map one to the other. If margin or cost is uncertain, the transformed range must not be presented as though those inputs were known exactly.

[![At forty-percent contribution and thirty thousand dollars of incremental cost, break-even revenue is seventy-five thousand; three illustrative study intervals fall below, cross or exceed that boundary.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-threshold.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-threshold.png)Compare evidence with the economic threshold, not just with zero incremental revenue.## Explain uncertainty without turning it into either paralysis or certainty

An interval crossing zero does not prove the effect is zero. It means the analysis has not separated the estimated effect from the relevant uncertainty at the reported level and under the method’s assumptions. The range may still contain effects large enough to matter commercially.

An interval excluding zero does not make the point estimate exact. It also does not remove implementation problems, outcome-definition errors or limits on where the result applies. Statistical precision is one part of credibility, not a replacement for a valid comparison.

Use the interpretation appropriate to the method. A frequentist confidence interval is not automatically a statement that this fixed effect has a ninety-five percent probability of lying inside the reported endpoints. A Bayesian credible interval has a different interpretation tied to its model and prior assumptions. Ask the analyst to explain which object is being reported in ordinary language.

The practical difficulty is well established. [Lewis and Rao’s research on advertising measurement](https://doi.org/10.1093/qje/qjv023) found substantial uncertainty even in large field experiments. The relevant lesson for an operator is that a large-looking dataset does not guarantee a precise answer about a small economic effect.

Separate statistical uncertainty from business uncertainty. The study may estimate revenue with a defined interval, while future margin, creative performance or available stock remains uncertain for other reasons. Do not compress all of those questions into one tidy error bar unless the analysis actually models them.

For Linden’s main example, the same ninety thousand revenue point estimate produces negative seven thousand five hundred dollars after advertising at a twenty-five percent contribution rate, six thousand at forty percent and nineteen thousand five hundred at fifty-five percent. This margin sensitivity is not a replacement for the study interval; it exposes a separate assumption.

A useful decision can still be made under uncertainty. The team might maintain a bounded activity while improving measurement, reduce exposure, or choose a different use of the budget. State which risk is being accepted. Do not describe a management preference as though the experiment uniquely dictated it.

The mistake to avoid is requiring certainty only from the option you dislike. If the current budget remains in place without strong evidence, while a reduction is rejected because its effect is uncertain, the decision process is asymmetric. Compare the uncertainties and consequences of both choices.

## Distinguish an unhelpful result from an invalid study

A result can be valid and commercially disappointing. It can also be too imprecise to distinguish relevant options. Neither makes the study broken. A failure of assignment, delivery or measurement is a different category and should be documented with evidence.

Suppose the activity was meant to be withheld in comparison markets, but another campaign delivered a similar intervention there. That can change what contrast was actually tested. The review should describe the contamination and its likely implications rather than silently reporting the planned design as the executed design.

Suppose a warehouse shortage affected the treatment markets but not the comparison markets. The effect on interpretation depends on the design and causal structure. It may be an implementation disruption, part of the real capacity constraint under higher demand or a competing event. Do not automatically remove the affected period simply because it lowers the estimated return.

Check whether the main outcome was observed consistently. If one group has a different missing-data pattern, the analysis needs to address it through the method’s assumptions and diagnostics. A better-looking attributed report cannot repair an experiment’s outcome measurement problem.

Distinguish the intervention assigned from the exposure received. Restricting the analysis after the fact to people who clicked can select a particularly responsive group and change the causal question. Use the analysis population and estimator appropriate to the study rather than making the result resemble a click report.

Consider interactions and spillovers. A household member can influence another buyer; advertising in one region can reach people who later purchase elsewhere. Those possibilities do not automatically invalidate every test, but they can limit the interpretation and should be part of the design review.

When the study is invalid for the intended decision, say exactly why and what remains usable. It may still document delivery problems or suggest a new design. Avoid converting an invalid causal estimate into a confident directional conclusion simply to avoid reporting that the question remains unanswered.

When the study is valid but inconclusive, keep that label. The next task is to decide whether better information is feasible and worth its cost, not to search the report for a favorable subgroup and declare the experiment successful.

## Choose a response that matches the evidence state

If definitions do not match, repair comparability before changing the spending story. The immediate action may be aligning revenue status, time windows or campaign scope. Keep the operational budget decision bounded while that work is completed.

If a credible result lies below the commercial threshold for the tested contrast, reconsider the activity or its cost structure. That might support reducing the tested scope, improving the offer or addressing an expensive execution model. It does not establish that the channel can never work in another form.

If the range crosses the threshold, write an uncertainty decision. Specify how much exposure the business is prepared to carry, how reversible the choice is and what further evidence would change it. A temporary bounded continuation can be reasonable, but it should have an owner and review date rather than becoming permanent by inertia.

If the evidence supports the tested activity above the threshold, preserve the scope of that conclusion. A presence-versus-absence result can support keeping the activity. A budget-increase result can support the tested increment. Further expansion changes the decision and may encounter different audiences, auction prices or fulfillment capacity.

If the study is invalid, repair the design or measurement before using its effect estimate as a budget rule. It may be prudent to limit exposure while resolving the issue, but the reason is uncertainty and operational risk, not a causal conclusion the data did not support.

Keep short-term operating decisions distinct. A broken landing page should be repaired even while the incrementality question is unresolved. A sudden event-tracking failure needs investigation. Those actions do not require declaring the entire channel profitable or unprofitable.

Document alternatives honestly. Stopping an activity does not automatically create an equally effective replacement. Keeping it does not make its opportunity cost disappear. The decision should compare the feasible next uses of money and team attention under the evidence available.

Most importantly, name one action. A meeting that ends with several incompatible interpretations and no owner has not used the study. The action can be maintain, reduce, change, investigate or retest, but it should specify the scope and the reason.

[![Decision responses differ for incomparable definitions, invalid execution, evidence below the commercial threshold, a range crossing the threshold and evidence supporting the tested scope.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-decisions.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-decisions.png)The evidence state should determine the next question and action, not just the dashboard color.## Do not force attribution to reproduce one experiment

A tempting shortcut is to divide estimated incremental revenue by attributed revenue and apply the ratio to future reports. In Linden’s main example, ninety thousand divided by one hundred eighty thousand is one-half. That arithmetic can be recorded as a descriptive comparison for that scope.

It is not automatically a stable correction factor. The attribution window, audience mix, season, creative, spend level and surrounding channels can all change. The experiment estimate itself has substantial uncertainty. Multiplying every future attributed dollar by one-half can create false precision while hiding those changes.

A carefully designed calibration process may use experiments as evidence, but it needs a stated model, uncertainty treatment, scope and refresh logic. A single ratio copied into a spreadsheet is not that process. Do not call the adjusted number causal merely because an experimental result appeared somewhere in its history.

Changing the attribution model until it matches the study is similarly risky. Several rules can produce the same aggregate credit while assigning it differently across campaigns and customers. Matching one total does not validate the underlying journey assumptions.

Keep the two views visible. The operating view can show attributed outcomes under a stable definition; the evidence view can show what the experiment estimated for a particular intervention. Their relationship is something to examine, not a discrepancy to erase at all costs.

When a platform provides a supported experimental metric, use its stated definitions. [Google’s geo-based Conversion Lift reporting guidance](https://support.google.com/google-ads/answer/14102986?hl=en), for example, distinguishes incremental conversion value, incremental cost and the reported uncertainty. Those metric definitions still need to be connected to the brand’s own contribution and decision scope.

A useful operating rule is to change the decision belief before changing the reporting definition. If the study weakens confidence in a campaign’s commercial contribution, document that belief and the resulting action. Do not rewrite historical attribution so the old dashboard appears to have known the answer all along.

## Use subgroup findings to form questions, not rescue the headline

After an inconclusive result, a team may search through markets, devices, products and audience segments until one looks positive. Some variation can be informative, but searching many slices creates opportunities to find apparently impressive patterns by chance.

Distinguish planned subgroup analysis from exploratory analysis. A prespecified question with an appropriate method has a different evidential status from a pattern discovered after reviewing dozens of charts. Label the latter as a hypothesis unless further analysis supports a stronger claim.

Ask whether a subgroup difference is itself supported, rather than assuming that one subgroup being statistically significant and another not significant proves they differ. The relevant comparison requires its own analysis. Unequal sample sizes can create different precision without establishing different effects.

Commercial plausibility still matters. A segment may show positive revenue but have lower contribution, higher service costs or limited stock. The next decision should use the same economic discipline as the overall result, not waive it because the subgroup offers a more encouraging story.

If a follow-up study is proposed, identify what it could resolve. A better-focused audience, a cleaner intervention or improved outcome capture may provide more useful information. Repeating the same underpowered setup merely because the first answer was unwelcome is not a learning strategy.

Do not promise that doubling the duration will solve every uncertainty. Seasonality, carryover, operating changes and design structure can complicate a longer study. The analyst should assess the proposed design using the actual data and decision threshold.

Retesting has a cost beyond the analysis fee. It can withhold useful advertising, require additional spend or delay a commercial choice. Compare the value of improved information with those costs and with a reversible operational change that may be possible sooner.

## Write a decision memo that keeps the limits attached

Linden’s memo begins with the tested contrast: the defined activity present versus absent in the specified population, with the surrounding marketing arrangement retained. It records the observation period and mature net-revenue outcome. This prevents a later reader from treating the study as a universal verdict on the channel.

The memo records the attributed view separately: one hundred eighty thousand dollars credited under the selected rule and thirty thousand dollars of relevant cost. It names the window and scope rather than showing an unexplained six-to-one ratio.

The experimental view follows: ninety thousand dollars estimated additional revenue with the supplied interval, plus the implementation checks and remaining limitations. The memo states that this is an estimate for the intervention, not a list of individual incremental orders.

The commercial section shows the forty-percent contribution assumption, seventy-five thousand dollar revenue threshold and the mapped range crossing that threshold. If the margin estimate is uncertain or sales mix differs, it adds that limitation instead of allowing the arithmetic to imply more certainty than exists.

The action section does not say scale because lift is positive. It might say maintain the current bounded scope while resolving the contribution estimate and assessing a more decision-relevant follow-up, with a fixed exposure limit and review date. That is an illustrative management choice, not the only defensible response to the hypothetical data.

Another business with tighter cash constraints or a stronger alternative use of the budget might reduce exposure. The memo should make that preference visible. It should not claim that the experiment uniquely requires either continuation or reduction when the commercial range spans both outcomes.

List the reopening conditions. A materially different margin, a verified implementation flaw, a new capacity constraint or stronger follow-up evidence can change the choice. Ordinary daily fluctuations in attributed ROAS should not automatically overwrite the agreed interpretation.

Finally, assign responsibility for execution and verification. A decision to reduce a defined activity must be checked against actual delivery and spend. A decision to keep it temporarily must have a review owner. Measurement creates value when it changes a controlled action and the business learns from the result.

[![A decision memo preserves the tested contrast, separate attributed and experimental views, contribution threshold, bounded action, owner and conditions for reopening the decision.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-memo.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A08-memo.png)Keep the assumptions and limits next to the action so that the result is not overextended later.## Do not use an average effect as the return on the next dollar

Imagine that a presence-versus-absence study supports keeping a campaign at its current level. The business then proposes doubling spend. The new decision changes the audience reached, delivery frequency, auction conditions and possibly creative mix. The first study does not directly estimate that new contrast.

A campaign can have attractive average contribution across its current activity while the next increment is weak. It may already reach the most responsive eligible customers. Additional delivery can become more expensive or less productive. The opposite is also possible when a small starting budget prevented a coherent campaign from operating, so the direction should not be assumed without evidence.

Consider a separate invented increase-versus-baseline result. Baseline spend is twenty thousand dollars and treatment spend is thirty thousand. The relevant extra cost is ten thousand. If the supported estimate of extra revenue is twenty thousand, incremental revenue per extra dollar is two, not twenty thousand divided by the treatment’s full thirty thousand dollar spend.

At a constant forty percent contribution rate, that extra revenue produces eight thousand dollars before the added advertising cost and negative two thousand after it. These simplified point calculations omit uncertainty to explain the denominator. A real decision still needs the study’s range and implementation checks.

This result could support rejecting that specific budget increase while leaving the baseline activity intact. It would be a mistake to infer that the whole campaign should be stopped, because the study did not compare baseline activity with absence. The decision statement should preserve exactly which spend contrast was evaluated.

The same discipline applies to reductions. Removing a campaign can change the role of other channels, not just subtract a fixed slice of credited revenue. Keep the surrounding marketing conditions attached to the estimate. A model that assumes every budget dollar has the same return can be convenient, but convenience is not evidence that the assumption holds.

## Keep the operating review and the evidence review on different clocks

Advertising teams need to respond to daily problems. Experiments often require a longer, planned observation period. Trying to make every daily decision wait for a new study is impractical; treating every daily attribution fluctuation as a new causal result is equally misleading.

Define a small set of operating guardrails that can trigger action without changing the causal conclusion. Examples include a verified tracking outage, a broken destination, an unauthorized spend change or a product becoming unavailable. The response should address the observed problem and record its effect on any ongoing analysis.

Use a separate review for beliefs about commercial contribution. That review considers mature outcomes, study validity, uncertainty and whether the business context still resembles the tested scope. A week’s lower click-attributed ROAS can prompt investigation, but it does not by itself erase a credible experiment or prove the campaign’s causal effect has vanished.

Conversely, a successful old study should not become permanent protection against new evidence. Product economics, audience saturation and customer behavior can change. State conditions that would make the result less transportable, such as a major offer redesign or a large expansion into unfamiliar markets.

This separation helps the team avoid oscillation. Without it, a positive experiment can trigger expansion on Monday, a noisy attribution report can trigger cuts on Wednesday, and the resulting delivery changes can make Friday’s comparison impossible to interpret. A stable decision record is not an excuse to ignore reality; it is a way to distinguish a relevant change from ordinary noise.

Keep a log of the actions actually executed. A recommendation to reduce spend is not an executed reduction, and a configuration change is not proof of the intended delivery difference. Verification should show what happened, when it happened and which future review can reasonably assess the result.

## Ask whether more information can change the choice

An inconclusive result does not automatically justify another expensive study. Start by naming the decision uncertainty that matters. If every plausible commercial outcome favors the same feasible action, a narrower statistical interval may be interesting without being operationally valuable.

For Linden’s main example, the range crosses break-even. Better information could matter because it may change whether the activity is kept at the tested scope. But first check whether the contribution assumption is the dominant unresolved issue. If the business cannot estimate its relevant margin, another revenue experiment may leave the same decision ambiguity in place.

Compare ways to learn. Improving return-cost accounting, repairing outcome coverage or validating the actual spend contrast may be cheaper and more useful than repeating the entire experiment. Those improvements answer different questions, so they should not be described as substitutes for causal evidence when they are not.

Consider reversibility. A modest temporary change with a clear limit may allow the business to manage exposure while preparing a better study. A large irreversible commitment deserves a different evidential threshold. The same interval can lead to different decisions when the consequences of being wrong are different.

Do not calculate the value of information by assuming the next study will reveal the truth perfectly. It will have its own uncertainty and implementation risks. Ask the analyst what outcomes are realistically distinguishable and how each would change the action. If the answer is that every result leads to continuing the current plan, the proposed study has no clear decision purpose.

End the discussion with a learning contract: the unresolved question, the evidence to obtain, its owner, cost boundary and deadline, and the actions attached to materially different findings. That contract turns uncertainty into work that can be completed rather than a permanent reason to defer judgment.

One final check is whether the proposed action can be observed well enough to review. If the team changes several channels, the offer and the website together, a later sales change may not isolate the effect of the specific budget choice. That does not prohibit coordinated business changes, but the review must describe the combined intervention rather than crediting one convenient component.

For a bounded operational adjustment, record the stable comparison metrics and known concurrent events before execution. Use the resulting review as operational evidence unless the design supports a causal claim. A disciplined change log improves learning without turning every before-and-after chart into an experiment.

Keep canceled plans in the record as canceled. If a study recommendation was approved but never implemented, later performance cannot validate the recommendation. Likewise, if actual delivery was materially smaller than the intended contrast, the interpretation needs to reflect that difference. Clear execution status protects the team from building future decisions on an action that existed only in a presentation.

The final reviewer should be able to trace the argument from observation to estimate, from estimate to commercial assumption, and from assumption to action. Missing links in that chain are the next work to do.

## Questions about attribution and incrementality disagreements

### Should an incrementality result replace the attribution dashboard?

Usually it should inform a different layer of decisions. Attribution can remain useful for consistent operating diagnostics, while a credible experiment informs the effect of a defined intervention. Keep the scope and limitations visible rather than forcing either report to answer every marketing question.

### Does no significant lift mean the campaign does nothing?

No. It may mean the study could not distinguish the effect from uncertainty at the reported level. Inspect the interval, design and economic threshold. A range containing commercially important effects is different from evidence that tightly rules them out.

### Can a positive experiment still support reducing spend?

Yes. Positive additional revenue may not cover the relevant advertising and operating costs. Also consider what contrast was tested, whether the result applies to the proposed change and what feasible alternatives exist. Positive effect and attractive commercial return are separate conclusions.

### Can we apply one experiment’s correction ratio to all campaigns?

Not without a defensible calibration method and scope. A ratio can change with audience, spend, season, measurement rules and the surrounding channel mix. Preserve experimental uncertainty and avoid presenting a single historical comparison as a universal causal multiplier.

### What should the team do immediately when reports disagree?

Write the decision, align scope and definitions, inspect study validity and place the estimated effect beside the commercial threshold. Then choose a bounded action with an owner. Do not begin by changing attribution settings until the two headline numbers look similar.

For channel-specific interpretation, continue with [brand-search demand](https://dtcup.com/blogs/insights/paid-brand-search-existing-demand-incremental-value/), [social and video evidence](https://dtcup.com/blogs/insights/social-video-click-view-attribution-incrementality/) and [email and SMS contribution](https://dtcup.com/blogs/insights/email-sms-attribution-holdout-contribution/). For designing the study itself, use [the incrementality test audit](https://dtcup.com/blogs/insights/incrementality-testing-for-acquisition-a-practical-audit/). The purpose of combining them is a better decision, not a single number that conceals every uncertainty.



 ## Sources

1. [doi.org — qjv023](https://doi.org/10.1093/qje/qjv023)
2. [support.google.com — 14102986](https://support.google.com/google-ads/answer/14102986?hl=en)

CONTINUE / THE TOPIC

## Keep the next question connected.

[NEXT BEST READ**Run a Marketing Measurement Meeting That Ends with One Useful Decision**](https://dtcup.com/blogs/insights/marketing-measurement-review-decision-method/)[RELATED GUIDE**Paid Search Attribution: Separate Brand Demand from New Demand**](https://dtcup.com/blogs/insights/paid-brand-search-existing-demand-incremental-value/)[RELATED GUIDE**Paid Social and Video: What Click Attribution Misses—and Views Cannot Prove**](https://dtcup.com/blogs/insights/social-video-click-view-attribution-incrementality/)[RELATED GUIDE**Email and SMS Attribution: Stop Giving Every Returning Sale to the Last Message**](https://dtcup.com/blogs/insights/email-sms-attribution-holdout-contribution/)

Still have a question?[Ask the DTCUP Community ↗](https://dtcup.com/community/)
