How to review a marketing move with AI without asserting cause

Signups rose the week after the landing page was fixed. CPA fell in the month the ad text was changed. When the numbers improve, it is tempting to report that "the move worked."

If budget, season, traffic source, price, or measurement changed in the same period, that move alone cannot be called the cause. The reverse is also true. Numbers that did not move do not, by themselves, mean the move was pointless. Some moves take time to show up, and some cases have little data.

When you review a move with AI, before you let it decide success or failure, have it separate the change that was observed from the cause that is not yet proved. This article introduces the fields to record and the prompt for that.

"It increased" and "the move increased it" are different

Comparing before and after a move tells you that the numbers changed in the same period. To check that the move caused the change, you also need a comparison with a condition where the move was not made.

Google Ads experiments split traffic or budget between the original campaign and the changed conditions and compare them. About experiment results also lets you check confidence intervals and statistical significance when you read the result.

Not every move can be a strict experiment. In an ordinary before-and-after comparison, change how the conclusion is worded, like this.

The aim is not to soften every sentence. It is to leave a record of what to check next so the judgment can get stronger.

Record six things before the review

Before you hand AI only the numbers, line up these six items.

1. The move and the date: what you actually changed, and when
2. The KPI you aimed at: conversions, CPA, CTR, search traffic, or whichever metric you expected to change
3. The baseline period: the comparison period before the move, and the numbers
4. The result period: a period of the same length after the move, and the numbers
5. What else was happening at the same time: budget, other moves, seasonality, measurement, outside factors
6. The state of the data: gaps, waiting for data to land, a small base

If you had not chosen a KPI before the move, write that fact too. Picking only the number that improved the most afterward makes the review convenient for the move.

A longer period is not always better. A move that shows up quickly, such as ads, and a move that takes time, such as SEO, have different times to check. If weekdays or holidays affect the numbers, use periods that include the same number of each weekday.

An input template for ChatGPT or Gemini

Enter one move at a time, in this shape. If you review several changes together, you cannot tell which change to keep next.

Move to review:
- Name of the move:
- Change that was made:
- Date:
- KPI this move was meant to change:
- Target or decision rule set before the move:

Before:
- Period:
- KPI value:
- Count that is the denominator (impressions, clicks, sessions, and so on):

After:
- Period:
- KPI value:
- Count that is the denominator:

What else was happening at the same time:
- Budget change:
- Other moves:
- Seasonality, day of week, closed days:
- Change in how results are measured:
- Outside factors such as competitors, weather, news:

Notes on the data:
- Gaps, or waiting for data to land:
- A period that is not the usual one:
- Other:

If the KPI you compare is a rate, pass the numerator and the denominator as well as the rate. "CVR doubled" is a different strength of judgment when 1 became 2 than when 100 became 200.

A review prompt that does not assert cause

After the input template, paste these instructions.

You assist with reviews of marketing moves.
Use only the information you were given, and separate observed results from cause and effect.

Rules:
- Do not add events, numbers, or industry averages that are not in the input
- Even if the KPI changed after the move, do not assert that the move was the cause
- Classify numbers as "improvement observed", "no change", "worsening observed", or "not enough data"
- If you cannot decide whether a metric moved in a good or bad direction, label it "change only"
- If the baseline value or the result value is missing, label it "not enough data"
- When you evaluate a rate, also check the numerator and denominator counts
- Keep things that happened at the same time as factors that could be another explanation
- Do not use a KPI that was not chosen before the move as a reason for success after the fact
- If no statistical test was run, do not write that there is a significant difference

Output:
1. A summary of the move, the date, and the KPI it aimed at
2. A table of the numbers before and after, and the change
3. The classification of the observed result, and the basis for it
4. What can be said about cause and effect, and what cannot
5. Factors that were present at the same time, and the chance they affected the result
6. Information that is still missing for a judgment
7. One small test to run next
8. A 3-sentence summary for a manager that avoids assertion

Check the AI output in four places

Check the text it returns in this order.

1. Are the periods comparable?

See whether it compared 7 days before with 30 days after as they stand, and whether the weekday mix is the same. If the period is still being measured, check that the text says so.

2. Is the size of the change calculated correctly?

AI sometimes gets a ratio wrong. Recalculate the original numbers and the rate of change in a spreadsheet. For a rate, look at the numerator and the denominator too.

3. Have changes from the same period disappeared?

If budget and conversions both rose, check CPA as well as conversions. If it was a sale period, it may not repeat in an ordinary month.

4. Is the next test narrowed to one?

The point of the review is to improve the next decision, not to explain the past neatly. If several changes are proposed, narrow them to the one that reduces the largest uncertainty.

Keep the move and the result in the same place

This review can be done with a free AI, but you still have to collect the date, the baseline at the time, and later changes from separate places. If you keep only the moves you ran, and not the reason you passed on others, you will consider the same idea again and again.

In Shoryaku AI, proposed moves can be split into "do this week," "hold," and "skip," and you can record the date and the KPI to check. For a move that was run, it lines up the baseline period and the result period 14 days later and 28 days later, and shows the observed result as "improvement," "no change," "worsening," "change only," or "not enough data." Budget changes, other moves, seasonality, measurement changes, and outside factors can be recorded too. Cause and effect is not treated as settled.

If you want to sort the choice of move before you run it, How to ask AI for ideas that raise revenue is also useful.

Summary

When you review a marketing move with AI, do not have it write a success story. Make a record you can use for the next test.

1. Keep the date and the KPI you chose beforehand
2. Compare periods of the same length before and after
3. Pass the numerator and the denominator, not only the rate
4. Write budget, other moves, seasonality, and measurement changes alongside
5. Separate the change you observed from the cause you have not proved
6. Narrow the next change to try to one

Start by picking one move you ran recently, and write down three things: the date, the KPI you aimed at, and what else was happening at the same time. Line up these three before you evaluate the numbers, and the AI review moves from an impression to a check.

• Marketing
• AI
• Product