A higher conversion rate doesn’t mean the discount won.

Shopify can now test discounts with Rollouts. Useful. But if the winning version is just “the one that converted better”, you can still end up celebrating a worse commercial result.

Discount tests are not really about conversion rate.

They’re about what you were trying to achieve in the first place.

If you sell a product once and never expect to see that customer again, I care about gross margin and contribution from the test.

If the goal is customer acquisition and repeat purchase, then I’m much more interested in projected lifetime value.

The same discount can be a terrible idea for one business and a brilliant idea for another.

Discounts can train customers too

It’s very easy to cannibalise your own sales with uncontrolled promotions.

Keep running discounts with no plan and you can end up in a death loop where customers stop expecting to pay full price.

They wait.

They learn your patterns.

They know another code is coming.

That doesn’t mean discounts are bad.

A well-timed discount can clear excess stock, reactivate a slowing product, create a reason to talk about the range, bring dormant customers back or support a wider commercial objective.

The problem is not discounting.

The problem is discounting without knowing why you’re doing it.

“Random” is only random to the customer

I’ll happily run a promotion that looks random from the outside.

Internally, there should still be a reason.

Maybe I need to move extra stock. Maybe a strong product has slowed. Maybe I want to create some social engagement or demand around a category.

But if you discount a random product every Tuesday, it stops being random.

You’ve trained the customer to come back on Tuesday.

So ask the obvious question: does your traffic increase on Tuesdays?

If it does, that behaviour is now part of the commercial result of the promotion — whether you intended it or not.

I want three levels of proof

For most commercial decisions, I like three levels of proof.

First: Shopify. It gives me the instant operational view. What changed? Did conversion move? Did order volume move? What happened to revenue?

Second: third-party analytics. I want another system checking the behaviour so I’m not relying on the platform that ran the experiment to mark its own homework.

Third: the bank account. Did the business actually make more money?

If all three are pointing in the same direction, I usually have my answer.

If they disagree, that is where the interesting work starts.

Shopify Rollouts gives merchants a much better way to test discounts than simply running a sale and comparing it with last week.

But the tool still doesn’t decide what success means.

That part is still your job.