# Email and SMS Attribution: Stop Giving Every Returning Sale to the Last Message

A message can receive credit for a purchase the customer already intended to make. Use audience holdouts, contribution and contact burden to decide which reminders actually deserve another send.

Source: https://dtcup.com/blogs/insights/email-sms-attribution-holdout-contribution/

DTCUP / FIELD NOTES / ACQUIRE

# Email and SMS Attribution: Stop Giving Every Returning Sale to the Last Message

A message can receive credit for a purchase the customer already intended to make. Use audience holdouts, contribution and contact burden to decide which reminders actually deserve another send.

Published Oct 2, 202625 min read

 ![Jack Hu](https://dtcup.com/wp-content/plugins/dtcup-knowledge/assets/jack-hu-small.jpg)**Jack Hu**7 years of experience building and operating brand websites, specializing in website development, paid advertising and SEO. Built a brand website from launch to US$10 million in annual sales.



  On this page  ## The Short Answer

Keep message engagement, attributed orders and incremental policy outcomes separate. Compare appropriately assigned eligible customers, use the same mature revenue and contribution definitions, and evaluate extra contact against customer burden rather than treating every matched returning order as new demand.

## Key takeaways

- Existing purchase intent makes lifecycle audiences different from the wider market.
- Changing event rules reallocates credit within an order cohort.
- A holdout must match the specific message policy being evaluated.
- Use assigned customers, not only clickers, for the main policy comparison.
- Adding SMS to email has a different marginal question from SMS alone.
- Higher purchase rate can coexist with lower contribution after discounts and contact cost.

![A brass balance holds ivory envelopes and a green speech shape opposite an orange cart token and a sage weight; another cart token sits beyond a glass divider.](https://dtcup.com/wp-content/uploads/2026/10/dtcup-a06-operating-guides-20261002-1024x576.webp)A customer has bought the same refill every month for a year. This month, a promotional email arrives two days before the usual reorder date. The customer clicks it and buys. The email platform assigns the order to that message. The attribution record may be correct under its rules, but a different question remains unanswered: would the customer have bought without the email?

Email and SMS often reach people who already know the brand, own its products and have a reason to return. That makes them valuable channels. It also makes them easy to overcredit. A message can help complete a purchase, remind someone of a need, change the timing, reduce friction or do nothing beyond appearing near an order that was already likely.

I would keep three views together: message delivery and engagement, attributed orders under a stated rule, and incremental outcomes from an appropriate comparison. Each view supports a different decision. Removing attribution because it is not causal loses useful diagnostics. Treating attribution as proof of additional sales can encourage unnecessary discounts and excessive contact.

This guide uses a fictional household-products brand, Cedar House. All campaign results, audiences, costs and test outcomes are constructed illustrations. They are not customer results or recommended universal benchmarks. The goal is to show how a team can move from an impressive message report to a defensible operating decision.

## Decide which question the report must answer

Start with the action under consideration. Should the team send another weekly campaign? Keep a replenishment reminder? Add SMS after an email? Change the attribution window? Increase a discount? These decisions require different evidence, even when they all appear on the lifecycle marketer’s dashboard.

Delivery data helps determine whether a message reached its intended technical destination. Engagement data shows recorded interactions such as clicks. Attribution assigns eligible purchases to interactions or deliveries according to a model and window. An experiment estimates how outcomes differ under alternative messaging policies, subject to the design and uncertainty.

A high click rate can indicate that the offer or content attracts attention. It does not show how much extra profit the campaign created. A high attributed revenue figure can help locate messages associated with purchase completion. It does not reveal the baseline purchase rate of the people selected to receive them.

Write the reporting question next to the metric. For example, which message links are being used is a diagnostic question. How many matched orders meet our seven-day click rule is an attribution question. Does adding a second reminder increase contribution per eligible customer is an incremental decision question. The wording prevents the team from answering the third question with evidence collected for the first.

Do not demand that one report perform every role. A campaign operator needs fast delivery and link diagnostics. A finance review needs consistent money definitions. A program decision may need a longer comparison period. The useful system connects those views without pretending they have the same clock, denominator or interpretation.

The broader [channel-by-channel attribution framework](https://dtcup.com/blogs/insights/choose-attribution-model-channel-decision/) explains this separation across marketing. For email and SMS, the special challenge is that the reachable audience is often selected for existing interest. That selection must remain visible in the analysis.

## Recognize the demand already present in the audience

A replenishment flow targets people who have purchased a consumable. A cart reminder targets people who already placed something in a cart. A loyalty campaign targets existing customers. These are not random samples of the whole market. They contain signals of demand before the message is sent.

Suppose Cedar House sends a reminder to recent refill buyers and sees an eight percent purchase rate. Another campaign goes to old subscribers with no purchases and produces one percent. The first message may be better, but the rates also reflect different audiences. Comparing them directly cannot isolate the effect of the creative, timing or channel.

The same issue appears inside an audience. Customers who click may have stronger current intent than those who do not. Comparing clickers with nonclickers after sending the message does not recreate a randomized comparison. Clicking is behavior that occurs after the intervention and may be associated with other reasons to purchase.

For a cart reminder, the baseline can be especially important. Some people leave checkout to find a card, ask a partner or finish another task, then return without needing a reminder. The message may still help others. The correct question is how much the reminder changes the outcome for eligible abandoned-checkout customers, not how many eventual buyers received it.

Define eligibility before exposure whenever possible. Record why the person entered the audience, the relevant time, current permission and suppression state, and the policy being tested. A comparison group should come from the same eligible population rather than from whoever happened not to receive the message for unrelated reasons.

Existing customers are not a reason to dismiss lifecycle marketing. They are a reason to measure it carefully. The channel may reduce uncertainty, improve product use and support repeat purchase. Its value should be demonstrated through the decision it changes, rather than inferred from the fact that loyal customers continue buying nearby.

## Build a small ledger before comparing revenue numbers

Cedar House begins with an order cohort: purchases placed during a specified week in the store’s reporting timezone. The team records the chosen revenue basis, refunds included as of a stated review date, and whether tax, shipping and discounts are included. This creates a stable commercial reference before any channel receives credit.

For each order, the attribution ledger records the matched customer or other supported identifier, eligible message events, event times, channel, campaign or flow, window and credit rule. It also records when no reliable match exists. A missing match should not be filled by assuming that the most recent campaign caused the purchase.

Suppose the cohort contains one thousand orders. Under a fictional broad message rule, four hundred qualify for email credit. A stricter click-only rule qualifies two hundred and fifty of those same orders. The remaining one hundred and fifty qualify only through other eligible events under the broader rule. These are different allocations of the same order cohort, not an increase or decrease in store sales.

Keep the comparison nested only when the definitions actually make it so. Changing both the event rule and the order dates can produce sets that are not subsets. The simple four-hundred versus two-hundred-and-fifty example assumes the same orders, identity coverage, money basis and window, with only the qualifying event changed.

Then inspect a sample of the difference. Are the additional orders associated with machine opens, genuine but unclicked messages, delivered SMS, or a configuration choice? The answer affects interpretation. Do not label all broad-rule orders fake, and do not label all of them incremental. The event record alone cannot settle that question.

When reports disagree, use the reconciliation approach in [why attribution reports differ](https://dtcup.com/blogs/insights/reconcile-attribution-report-differences/). Change one definition at a time. A controlled difference is informative; a collection of incompatible screenshots is not.

[![The same fictional cohort of one thousand orders contains four hundred broad-rule email orders, including two hundred fifty click-qualified orders and one hundred fifty additional event-qualified orders.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-ledger.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-ledger.png)Changing the qualifying event changes allocated credit. It does not create or remove orders from the store cohort.## Treat opens and clicks as evidence with limits

An email open is not always a direct observation of a person reading. Privacy and security mechanisms can affect recorded events. A click can also be generated or altered by automated systems. Even a verified human click establishes an interaction, not the counterfactual outcome without the message.

Klaviyo’s [attribution guidance](https://academy.klaviyo.com/en-us/quick-guides/understand-attribution) documents configurable event windows and filtering for bot clicks and machine opens. The practical lesson is to inspect the actual account configuration and its effective date. Do not assume that every account, channel or historical report uses the same settings.

For operational review, I would separate recorded events from the subset accepted by the reporting rule. Preserve the exclusion reason where the system supports it. If filtering changes, annotate the date and assess whether historical reporting is recalculated or only future attribution changes. Otherwise, a measurement change can be mistaken for a sudden decline in customer interest.

Click-only reporting can provide a clearer interaction threshold for some decisions, but it is not a complete causal model. A customer may read an email and later buy without clicking. Another may click only to access an account or compare a detail before making an already planned purchase. The same event type can play different roles.

Do not remove privacy-protected customers from every commercial analysis merely because an open signal is less interpretable. Their orders still belong in the relevant store cohort and an appropriately designed experiment. The uncertainty concerns a particular observation, not whether those customers exist or have value.

SMS needs its own interpretation. Delivery, link interaction and purchase are different events. A delivered message does not prove attention, and a purchase soon afterward does not prove persuasion. Keep email and SMS rules explicit instead of combining all touches into a generic engaged label.

The purpose of event hygiene is to make descriptive reporting more honest. It can reduce misleading associations. It cannot, by itself, tell the team how many customers would have bought in the absence of the message. That requires a different design.

## Choose a holdout that matches the actual policy decision

A holdout is a group intentionally excluded from the messaging being evaluated. The crucial phrase is being evaluated. Holding out one campaign, one flow, an extra SMS or an entire program answers different questions. Label the comparison accurately.

If Cedar House wants to know whether a second replenishment reminder adds value, both groups can receive the first reminder, while only the treatment group receives the second under the defined policy. The result estimates the additional reminder’s effect relative to the first-reminder policy. It does not estimate the value of all email marketing.

If the question is whether the whole promotional program adds value, a broader persistent holdout may be needed. Other marketing contacts, necessary service messages and exclusions must be defined. A group that receives fewer campaigns but continues to receive every major promotional flow is not a no-marketing comparison.

Klaviyo’s [global holdout documentation](https://help.klaviyo.com/hc/en-us/articles/18138290642971) describes a platform-specific implementation with eligibility requirements and channel scope. Treat those requirements as product details to verify, not universal rules for experimental validity. A custom campaign experiment and a vendor’s global holdout feature are not automatically equivalent.

Keep necessary transactional and service communications outside a promotional suppression test unless there is a separately justified, appropriate design. Customers still need the information required to use the service and manage their orders. The test specification should name what continues in both groups so that the measured contrast is understandable.

Random assignment helps make the groups comparable at the start. Use a stable eligible unit, such as a customer account, rather than randomly assigning every message independently when customers can enter several flows. Otherwise, one person may be in treatment for one message and control for another, and the claimed policy contrast becomes unclear.

Before launch, define the primary outcome, observation period, minimum useful effect, expected sample needs, exclusions, contamination checks and decision date. Do not choose these after seeing which metric looks favorable. A small brand may not have enough observations for a precise answer quickly; that limitation should shape the decision rather than be hidden by a confident dashboard label.

## Read a worked holdout without overstating what it proves

In a fictional Cedar House test, twenty thousand eligible customers are randomly assigned before sending: ten thousand to the defined promotional policy and ten thousand to a holdout without those promotional messages. Necessary service messages continue under the same rules. The fixed observation period and customer-level purchase outcome are specified beforehand.

At the review date, six hundred treatment customers and five hundred holdout customers have purchased at least once. The purchase rates are six percent and five percent. The observed difference is one percentage point, equivalent to one hundred additional purchasers per ten thousand assigned customers at the point estimate.

Notice the denominator: all assigned eligible customers in each group. It is not delivered messages, openers or clickers. Some treatment customers may not receive a message because of later suppression or delivery failure. Those events are part of the practical policy outcome and should be investigated, but removing them selectively from the main analysis can break the original comparison.

For illustration, an approximate independent-proportions standard error is the square root of six percent times ninety four percent divided by ten thousand, plus five percent times ninety five percent divided by ten thousand. That is about three tenths of a percentage point. A simple normal approximation gives a ninety-five percent interval of roughly zero point three seven to one point six three percentage points around the observed difference.

This interval is a calculation for the constructed independent customer-level example. Real analysis may require a different method because of clustering, repeated exposure, unequal assignment, missing outcomes or the randomization design. It also does not account for every possible implementation flaw. A precise formula cannot rescue a contaminated or incorrectly joined experiment.

Suppose the email platform attributes three hundred and fifty treatment purchasers to messages. That number can coexist with an experimental point estimate of one hundred additional purchasers. The first follows a touch rule; the second compares group outcomes. The difference is not proof that exactly two hundred and fifty named purchasers were falsely attributed. The experiment estimates an aggregate effect, not a list of individual counterfactual buyers.

Likewise, dividing one hundred by three hundred and fifty does not produce a permanent correction factor for every future campaign. The relationship can change with audience, offer, season, contact pressure and the rest of the marketing mix. Use the experiment to inform the tested policy and its plausible extensions, not to manufacture a universal incrementality percentage.

Unequal group sizes need normalization. In a separate illustrative allocation, suppose eighteen thousand customers receive the policy and two thousand enter the holdout. If purchaser counts are one thousand and eighty versus one hundred, the rates are still six percent versus five percent. Subtracting the raw counts gives nine hundred and eighty, which is not the estimated added purchaser count. At the holdout rate, eighteen thousand customers would correspond to nine hundred purchasers; the treatment point estimate is therefore one hundred and eighty above that comparison.

This rescaling is arithmetic, not a substitute for uncertainty analysis. A smaller holdout usually provides less precise information about its underlying rate than a larger one, all else equal. The chosen allocation should balance learning needs and the practical cost of withholding the tested policy. Do not assume that a familiar five-percent holdout is adequate for every baseline rate, audience size and minimum useful effect.

[![A fictional randomized policy test has six hundred purchasers among ten thousand treatment customers and five hundred among ten thousand holdout customers, a one percentage point observed difference.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-holdout.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-holdout.png)Use all assigned eligible customers for the main policy comparison. Platform-attributed purchasers answer a separate question.## Move from extra purchases to contribution

An extra purchase is not automatically an economically attractive purchase. Discounts, product mix, fulfillment cost, returns and contact cost can change the conclusion. A campaign can raise conversion while reducing contribution per eligible customer.

Continue the fictional example with a simplified assumption: every purchaser makes one order, so there are six hundred treatment orders and five hundred holdout orders. Treatment orders average one hundred dollars in net merchandise revenue and thirty dollars in contribution before message cost. Holdout orders average one hundred and ten dollars in revenue and thirty eight dollars in contribution.

The treatment group produces sixty thousand dollars in revenue and eighteen thousand dollars in contribution before message cost. The holdout produces fifty-five thousand dollars in revenue and nineteen thousand dollars in contribution. Revenue is five thousand dollars higher, but contribution is one thousand dollars lower before the extra contact cost.

If the treatment policy costs six hundred dollars more to deliver during the test, its contribution after that incremental cost is seventeen thousand four hundred dollars, versus nineteen thousand in the holdout. The observed difference is negative sixteen hundred dollars, or negative sixteen cents per assigned customer.

These are constructed totals, not a claim of statistical certainty about profit. Customer-level contribution has its own variation and may need a suitable uncertainty analysis. The example demonstrates why more purchasers and higher attributed revenue can still be consistent with a poor economic outcome.

In a real ledger, do not multiply all extra purchasers by one average margin if the policy changes basket composition or discounts across purchases that would have happened anyway. Compare the defined contribution outcome across the assigned groups. That captures both additional orders and margin changes on existing demand.

Be explicit about which costs are included. Variable message charges belong in the incremental policy cost when they change with the treatment. A platform subscription that remains unchanged may be relevant to an overall program decision but not to the marginal cost of one extra send. Avoid subtracting the same fixed cost twice or excluding a variable discount because it appears in another team’s budget.

[![In a constructed equal-size test, treatment revenue is sixty thousand dollars versus fifty-five thousand, but contribution after extra message cost is seventeen thousand four hundred versus nineteen thousand.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-contribution.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-contribution.png)Higher revenue can coexist with lower contribution when the message changes discounts, basket economics or contact cost.## Test the extra SMS without giving both channels the whole order

When email and SMS operate together, channel reports can each appear persuasive. The same customer may receive an email, click a text and later return through search. Allocating credit differently changes the story, but it does not establish whether adding SMS improved the email policy.

A useful design for a specific decision can compare four assigned policies: neither tested promotional channel, email only, SMS only, and both. This requires sufficient eligible customers, appropriate permissions for the included channels, and an implementation that actually maintains the intended differences. It is not a reason to contact people who are not eligible.

Imagine four fictional groups of five thousand customers. The purchaser counts are two hundred with neither, two hundred and fifty with email, two hundred and thirty with SMS, and two hundred and sixty with both. Their purchase rates are four percent, five percent, four point six percent and five point two percent.

Email adds one percentage point relative to neither in this sample. SMS adds zero point six percentage points relative to neither. But adding SMS when email is already present changes the rate from five to five point two percent: only zero point two percentage points, or ten purchasers per five thousand at the point estimate.

The observed combined effect of one point two percentage points is smaller than the sum of the separate one-point and zero-point-six effects. The interaction contrast is negative zero point four percentage points in this constructed example. This does not prove harmful interference without uncertainty analysis, but it shows why standalone channel effects should not simply be added.

The operating decision is the marginal one: does SMS add enough value on top of the email policy to justify its cost and contact burden? The answer may differ by audience, offer or timing. A broad claim that SMS works because its standalone group beats neither is insufficient for the additional-channel decision.

Do not run a complex four-group test merely because the diagram is elegant. If sample size is limited, a focused comparison of email versus email plus SMS may answer the immediate decision more clearly. Choose the smallest design that identifies the contrast you need, and acknowledge the questions it leaves open.

[![Four fictional groups of five thousand show purchase rates of four percent for neither channel, five percent for email, four point six percent for SMS and five point two percent for both.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-channel-mix.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-channel-mix.png)The relevant comparison for adding SMS is both versus email, not SMS versus no promotional contact.## Look for orders moved earlier rather than permanently added

A reminder can accelerate a purchase that would otherwise occur later. That may be useful for cash timing or convenience, but it is different from increasing total demand. A short observation window can confuse the two.

Suppose Cedar House sees a strong treatment advantage during the first week, followed by weaker repeat buying in the treatment group over the next month. One explanation is that the campaign brought forward purchases. Another is changed product mix or customer composition. The team should examine the timeline and pre-specified longer-term outcomes rather than declaring victory at the first favorable date.

Plot cumulative purchasers, orders or contribution by assigned group over a suitable period. A gap that appears quickly and then closes has a different meaning from a gap that persists. The observation period should reflect the decision and typical purchase cycle, while recognizing that longer periods introduce more external changes and contamination opportunities.

Replenishment products deserve special care. If customers buy two months of supply during a discount, the campaign can increase current basket size and reduce next month’s orders. Comparing only the promotion week overstates the durable effect. Compare the broader customer economics, including stockpiling and discount cost, where the data supports it.

Keep attribution windows separate from experiment observation windows. A seven-day click rule determines which touches receive credit. A six-week policy evaluation determines when group outcomes are assessed. They may legitimately differ because they answer different questions. Changing one does not automatically require changing the other.

Define how repeat orders enter the outcome before calculating the result. A purchaser-rate metric counts each assigned customer once, even if that customer places several orders. An orders-per-customer metric includes those repeats. Contribution per assigned customer includes the economic consequences of all included orders under the stated rules. A campaign that encourages stockpiling may improve one measure while leaving another unchanged. Those differences are information, not an invitation to select whichever metric makes the message look best.

Refund timing needs the same discipline. Compare groups using the same maturity cutoff and revisit the decision if a material share of returns is still unresolved. Do not compare fully settled holdout orders against fresh treatment revenue. For ongoing reporting, preserve both the initial read and the later mature read so that the team can see how the conclusion changed as the cohort developed.

Use the logic in [choosing an attribution window around the buying cycle](https://dtcup.com/blogs/insights/choose-attribution-window-buying-cycle/) to interpret delayed purchases. Recent cohorts need time to mature. A campaign sent yesterday cannot be compared fairly with one that has already accumulated a full month of outcomes.

## Put contact burden beside financial value

Email and SMS use a scarce resource: the customer’s willingness to keep hearing from the brand. The immediate cost of a send may be small, but repeated low-value contact can change future reach, complaints and the relationship. That cost does not appear automatically in a same-day revenue chart.

Record unsubscribe, complaint and suppression outcomes with clear denominators. A rate per delivered message is useful for message diagnostics. A rate per assigned eligible customer can be useful for comparing contact policies. Do not switch between them without explanation, especially when one policy sends more messages per person.

Frequency should be assessed at customer level across campaigns and flows. A marketer may see one reasonable campaign while the customer receives that campaign, a cart reminder, a product recommendation and two SMS messages in the same period. Separate teams can each follow their own schedule and create an excessive combined experience.

For a frequency test, define complete policies such as one weekly promotional contact versus two, including how conflicts with flows are resolved. Compare outcomes over the assigned population. A message-level test that ignores other contact may fail to measure the practical frequency change customers experience.

Do not assign an arbitrary large lifetime value to every unsubscribe and present it as measured loss. An unsubscribe is a real operational outcome, but its economic meaning depends on the customer, alternative channels and future behavior. Report the observed change, then use clearly labeled assumptions if estimating longer-term consequences.

Some guardrails should stop a rollout even when short-term revenue looks good. Define acceptable complaint and service-impact boundaries through the responsible business process before the test. The team should not discover its tolerance only after a high-revenue campaign damages reach or creates avoidable support work.

## Check whether the intended comparison actually happened

Random assignment is only the beginning. The sending system must respect it. A holdout can be contaminated by another campaign, a duplicated profile, an agency upload or a flow that bypasses the exclusion rule. If the groups receive similar contact, the experiment may estimate a much smaller contrast than the team intended.

Create an exposure audit using the channels in scope. For each assigned group, count eligible customers, attempted sends, delivered messages and relevant overrides. Review why holdout members received promotional contact and why treatment members did not. Keep the assigned-group analysis, while documenting deviations and their likely effect on interpretation.

Identity matters here too. If one person has several profiles, assigning each profile independently may expose the same person through different groups. Use the best justified, permitted unit available and state its limitations. Do not invent cross-device identity to make the experiment look cleaner than the records support.

Check outcome joins before interpreting a small lift. Both groups need comparable order capture. If treatment clicks improve identity matching while holdout orders are less likely to be linked, the measured difference can partly reflect observation rather than behavior. A store-side customer identifier may help when legitimately available, but unresolved coverage should remain visible.

Also inspect assignment balance and sample ratio. Large unexpected differences from the planned allocation can signal an implementation problem. Comparing pre-treatment purchase history and other relevant characteristics can reveal anomalies, although ordinary random differences will occur. Do not keep modifying the groups until the historical charts look identical.

Record major concurrent changes: promotions, prices, stock availability, site releases and other channel activity. Randomization can protect against shared changes when implemented properly, but differential exposure or stock constraints can change the policy being evaluated. If a product sells out because of treatment demand, the resulting interference deserves analysis rather than a casual footnote.

Finally, avoid repeatedly checking results and stopping at the first favorable point unless the statistical approach was designed for that process. A planned review date and method make the decision easier to defend. If a genuine operational problem requires early stopping, record the reason and interpret the evidence within that interruption.

## What a smaller brand can do when a precise holdout is unrealistic

A smaller audience may not support a precise experiment on a tiny incremental effect. That does not mean every attributed order should be accepted as additional revenue. It means the team needs a proportionate decision under uncertainty.

First improve descriptive consistency. Keep the order cohort, revenue basis, event rules and attribution window stable enough to compare. Inspect the actual messages and customer journeys associated with credit. This can reveal broken links, redundant reminders or discounts sent after a purchase without making a causal claim.

Second, focus on a meaningful policy difference. Testing two nearly identical subject lines may produce too little signal for the commercial question. A carefully scoped change to an unnecessary extra reminder may be more decision-relevant, provided the audience and service implications are appropriate.

Third, use uncertainty to size the action. A small reversible change with modest cost can be reasonable when evidence is directional. A large increase in discounts or contact frequency needs stronger support. The threshold should reflect the consequence of being wrong, not a blanket rule that every test must return a winner.

Longer observation can help accumulate outcomes, but it is not a free solution. Customer eligibility, season and the surrounding program may change. A persistent holdout must remain well defined, and the analysis should account for the design. Do not merge unrelated campaigns across months solely to reach a larger sample.

Customer feedback and service evidence can guide diagnosis. If people repeatedly complain that reminders arrive after buying, fix the event timing and suppression logic. That is an observable workflow defect. You do not need to claim a measured revenue lift to justify correcting it.

When no credible comparison is available, say what the report shows: matched orders associated with the defined messaging events. Use scenarios for incremental value rather than disguising an assumption as a result. An honest range can support a bounded decision better than a precise but unsupported return figure.

## Build a dashboard that leads to a specific change

I would organize the review into four connected panels. The first describes eligible customers and actual contact. The second shows engagement and attribution with the rules visible. The third shows the policy comparison and its uncertainty when available. The fourth shows contribution, customer burden and the next action.

For Cedar House’s constructed test, the action is not simply send more because purchase rate increased. The economic example shows higher revenue but lower contribution after contact cost. The team might test a lower-discount version, narrow the audience to a justified need state, or remove a redundant second message. Each is a new policy question requiring its own evidence.

The decision record should name the exact audience, message policy, comparison, dates and outcome. Include the attribution settings used for diagnostics, but do not let a change in those settings masquerade as the result of the campaign. Record what is uncertain and which observation would change the decision.

Separate a measurement repair from a marketing change. Filtering automated clicks may reduce reported engagement without changing any customer experience. Removing a message changes the experience. Both can be useful, but they should have different entries in the change log and different review criteria.

Save the audience definition and assignment record with the review. A segment that updates continuously can contain different people when someone reruns the report later. Preserve enough approved experiment metadata to reconstruct who was eligible at assignment, while following the organization’s data-handling arrangements. Reproducibility starts with the population, not just the final chart.

Assign the follow-up to a person and a date. For example, the lifecycle owner checks suppression and exposure integrity before another send, while the analyst reviews mature contribution at the planned endpoint. The campaign should not continue indefinitely because the original dashboard had a green arrow.

[![An email and SMS decision record connects eligible audience, actual contact, attribution rule, assigned-group outcomes, contribution, customer burden and the next bounded action.](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-decision.png)](https://dtcup.com/wp-content/plugins/dtcup-operating-guides/images/A06-decision.png)The final output is a policy decision with evidence and limits, not a contest over which platform gets the most credit.## Questions about email and SMS attribution

### Should we switch everything to click-only attribution?

Click-only reporting may suit a particular diagnostic standard, but it misses purchases without a tracked click and still does not prove incrementality. Inspect the actual event quality, account settings and decision first. Keep changes documented so that a reporting shift is not mistaken for a change in sales performance.

### Does a holdout show which individual orders were caused by email?

No. A well-designed comparison estimates an aggregate outcome difference under its assumptions. It does not identify every individual counterfactual purchase. Do not label specific attributed customers as incremental or nonincremental solely from the group-level result.

### Can I compare subscribers with people who never subscribed?

That comparison is usually affected by selection: subscribers may already differ in interest, purchase history and other ways. It can describe the groups, but it does not isolate the effect of receiving a message. A randomized comparison within an appropriately eligible audience is a different design.

### Why does an extra SMS look weaker when email is already running?

The channels may reach the same demand and provide overlapping reminders. The marginal effect of adding SMS to email can differ from SMS used alone. Compare the policies relevant to the decision, include contact cost and burden, and evaluate uncertainty before treating the observed difference as a durable rule.

### What should we report when the result is inconclusive?

Report the estimate, uncertainty, implementation quality and practical decision limits. Inconclusive does not mean no effect, and it does not justify choosing the most flattering attribution number instead. Keep the current policy, make a bounded reversible change or gather better evidence according to the cost of being wrong.

Email and SMS deserve neither automatic credit nor automatic skepticism. They deserve a clear question. Use attribution to understand recorded participation, a suitable comparison to estimate added value, and customer economics to decide what to do next. That combination protects the relationship while making the program more useful.



 ## Sources

1. [academy.klaviyo.com — understand-attribution](https://academy.klaviyo.com/en-us/quick-guides/understand-attribution)
2. [help.klaviyo.com — 18138290642971](https://help.klaviyo.com/hc/en-us/articles/18138290642971)

CONTINUE / THE TOPIC

## Keep the next question connected.

[NEXT BEST READ**Run a Marketing Measurement Meeting That Ends with One Useful Decision**](https://dtcup.com/blogs/insights/marketing-measurement-review-decision-method/)[RELATED GUIDE**Why Attribution Reports Disagree: Reconcile the Difference Before Blaming the Model**](https://dtcup.com/blogs/insights/reconcile-attribution-report-differences/)[RELATED GUIDE**Choose an Attribution Window Around the Buying Cycle**](https://dtcup.com/blogs/insights/choose-attribution-window-buying-cycle/)[RELATED GUIDE**When Attribution and Incrementality Disagree, Which Decision Changes?**](https://dtcup.com/blogs/insights/attribution-incrementality-disagreement-business-decision/)

Still have a question?[Ask the DTCUP Community ↗](https://dtcup.com/community/)
