Sales Fell, or Refunds Landed Late? What Your AI Trading Brief Must Explain

Your weekly report says sales fell 20%. The AI summary recommends reducing campaign spend. The store team says orders were steady.

All three statements can sound plausible. Only two may be supported by the records.

A reporting-period change is not automatically a demand change. If the agent cannot separate the two, it can turn a correct number into the wrong recommendation.

One week, three different questions

Take a fictional store reporting in Australian dollars, Monday to Sunday in Australia/Melbourne. To keep the arithmetic clear, exclude tax, shipping and discounts, and assume no cancellations or currency conversion. These are deliberately simplified operating measures, not accounting policy or universal Shopify report definitions.

In the previous week, the store recorded $50,000 of new orders and $2,000 of refunds processed during that week. Its chosen net trading measure was $48,000.

This week, it again recorded $50,000 of new orders. But it processed $11,600 of refunds: $2,000 relating to this week's orders and $9,600 relating to earlier orders.

The net trading measure fell by $9,600, or 20%. New order value did not fall. The additional processed refunds explain the arithmetic difference.

That does not make the refunds harmless. They may expose product quality, fulfilment or sizing problems in earlier orders. It does mean that this comparison alone does not establish weaker current demand or justify cutting this week's acquisition spend.

Now suppose bank payouts totalled $35,000 this week. That number answers another question: what settled into the bank during the window? Settlement timing, fees, reserves and earlier transactions may separate it from both order value and the net trading measure. Do not invent an explanation for the remaining difference. Match it to settlement records.

Define the measure before asking the agent why it moved

Our Shopify automation guide recommends a weekly trading brief that explains changes and proposes actions. This is one exception that brief must handle before the explanation becomes useful.

OpenAI's September 10 Data agent announcement describes analysis connected to company data and business definitions. It also says its own data team created shared definitions and access rules. The useful lesson is not that a product will reconcile your reports automatically. It is that the context behind a number is part of the analysis.

For each measure in the brief, agree:

• The business question: current order activity, refunds processed, cash settlement, or something else.
• The period: exact boundaries, timezone and which event date determines inclusion.
• The adjustments: treatment of refunds, discounts, cancellations, tax, shipping and currency conversion.
• The source: the report or records that supply the measure, extraction time and known reporting lag.

Have the operator and finance owner approve those definitions. Matching labels across systems is not enough: two fields called sales may include different events.

What the weekly brief should actually say

For this fictional week, a useful summary is specific about what is known:

New order value was unchanged at $50,000. Orders less refunds processed fell from $48,000 to $38,400. The difference is explained by $9,600 more refunds processed this week; $9,600 of this week's refunds relate to earlier orders. Investigate those refunded orders before attributing the decline to current campaigns. Bank payouts of $35,000 remain unreconciled to the settlement records.

It should link the underlying report and refund records for authorised reviewers. Separate observed changes from explanations and hypotheses. A batch of late refunds is observed; a faulty product batch is a hypothesis until the returned items and reasons support it.

Do not treat the current week's $2,000 of refunds as its final return rate either. Those orders have not all had time to be delivered, used and returned. Comparing order cohorts requires comparable time for returns to emerge.

Recommend an action, correct the data, or hold the conclusion

If refund records identify a specific product problem, the next step may be an investigation with merchandising or fulfilment. If source timestamps or definitions disagree, assign the correction to the data owner. If the evidence is incomplete, hold the affected recommendation and say exactly what is missing.

Holding one conclusion does not mean suppressing the whole report. The team can still see order activity, known refunds and unresolved cash differences. What it should not receive is a confident causal story that fills the gap.

Agree which unresolved discrepancies block which decisions. A payout mismatch may matter urgently to cash planning without invalidating the order count. Stale advertising data may block a spend recommendation without preventing investigation of a refund spike.

Automate a repeatable explanation, not a confident guess

A self-serve analysis tool may be enough when your definitions and records are already reliable. Additional help is valuable when the unfinished work is mapping sources, handling late adjustments and assigning recurring exceptions to an owner.

Before automating the weekly commentary, take one disputed report through this process manually. If the team cannot explain the difference with agreed definitions and traceable records, a more fluent summary will not solve it.