Don't Measure AI Visibility Once: Why You Need 7 Runs a Day and 2–4 Week Windows

Key takeaway: Brand visibility in AI search is not a value you can judge from a single check the way you would a search ranking. When University of St. Gallen researchers measured ChatGPT, Perplexity, Gemini, and Google AI Mode for 45 days, cited sources turned over roughly 65% from one day to the next, and even identical prompts re-run at the same moment overlapped only 32–43% (arXiv:2604.07585). Their convergence analysis sets a concrete bar: trust per-brand detection rates only at 7 runs per prompt per day, and per-brand trends only with 2–4 week rolling windows. A screenshot showing "we rank #3" may be a single point inside the noise (as of 2026-07-12).

Three-line summary

1. Most of the variation in AI search visibility comes from the probabilistic nature of LLMs, not from time — simultaneous re-runs still overlapped only 32–43% on cited sources.
2. The statistically defensible minimum is 7 runs per prompt per day (for brands) and 2–4 week rolling aggregation — comparisons made below that threshold are likely noise.
3. The practical conclusion: treat AI visibility as a distribution and a trend, not a ranking — via a diverse prompt portfolio, engine-specific baselines, and period-level reporting.

This is not a ranking game — brands vanish entirely

In traditional search, a ranking is a relatively stable value. A page that sits at #5 today may slip to #7 tomorrow, but it rarely disappears from the result set altogether. AI search behaves differently. The St. Gallen team calls this the "inclusion–exclusion" dynamic: brands and sources appear in one response and simply do not exist in the next.

The researchers ran eight prompts per campaign across four Swiss verticals (telecommunications, real estate, sporting goods, consumer electronics) against ChatGPT, Perplexity, Gemini, and Google AI Mode every day from January 24 to March 20, 2026. The day-to-day Jaccard similarity for cited sources averaged about 0.35 — meaning roughly 65% of cited sources were replaced within a single day. Rank-aware overlap (RBO) came in even lower at 0.21–0.26, so the ordering of the surviving sources shuffled as well. Brand mentions were more stable (45–59% day-to-day overlap) but still lost close to half of the set.

It is not about time — re-runs at the same moment overlap only 32–43%

If things change "overnight," you might suspect temporal factors such as model updates or refreshed sources. To isolate that, the researchers re-ran identical prompts at the same point in time — up to 10 runs per engine–prompt — and compared 3,409 pairs.

"Even when identical prompts are executed simultaneously under controlled conditions, cited sources and brand mentions vary substantially." — Schulte, Bleeker & Kaufmann, arXiv:2604.07585

Even simultaneous re-runs overlapped only 0.32–0.43 on cited sources — effectively the same range as the day-apart measurements. In other words, most of the observed instability comes not from the passage of time but from the probabilistic nature of LLM output itself. When yesterday's ranking differs from today's, it is often not because anything changed, but because you sampled once from a value that varies every time. The July 2026 Search Engine Journal report summarizing this line of research — "much of AI visibility ranking movement is statistical noise" — points at the same conclusion.

So how many measurements are enough — 7 a day, then 2–4 weeks

The study's practical contribution is that it does not stop at "measure repeatedly"; it computes how much repetition is enough through convergence analysis.

Value you want to trust	Required measurement	Basis (standard error)
Per-brand detection rate (probability of appearing that day)	7 runs per prompt per day	SE < 0.10 (95% CI ±0.158)
Cited-source coverage	8 runs per prompt per day	Source-level variance is higher, one extra run needed
Per-brand trend (period comparison)	Rolling 10+ days, 2–4 weeks recommended	SE < 0.10 at 10 days, < 0.05 at 24 days

Two more conditions apply. First, diversify prompts — per-prompt stability ranged from below 0.2 to above 0.8, so tracking one or two prompts measures the quirks of those prompts rather than campaign-level visibility. Second, separate baselines per engine — citation concentration differs meaningfully (Gini 0.782 for Google AI Mode vs. 0.671 for Perplexity), so a single threshold applied across all AI search products produces wrong conclusions.

In practice — distributions, trends, and period representatives instead of ranking screenshots

Translated into operations, the findings come down to three rules.

1. Never decide from "we appeared / we didn't." As the authors put it, visibility is "a distribution rather than a single-point outcome." Reporting "we rank #1" or "we dropped out" from one screenshot is like flipping a coin once and declaring the probability of heads.
2. Compare repeated measurements under identical conditions; report period representatives. Reacting to weekly trends and monthly representative values — rather than daily swings — means responding to signal instead of noise. The recommended 2–4 week rolling window also aligns naturally with a monthly executive reporting cadence.
3. Judge improvement work by the same rule. A rank moving the day after you improved content proves nothing. Compare before and after with repeated runs of the same prompt set to get anywhere near causality.

The same standard applies to tooling. RanketAI's AI brand visibility analysis runs multiple queries against multiple LLMs repeatedly to check mentions and citations, and the dashboard trends and monthly report present change on a period basis rather than a single point — the same direction this paper recommends. Whether you use Semrush AI Visibility, Ahrefs Brand Radar, or other repeat-tracking tools, the habit this research teaches is to ask first: "how many days and how many runs does this number aggregate?"

Frequently asked questions

Our brand came up #1 in ChatGPT yesterday — can I report that?

We would not recommend it. This study measured that even identical prompts re-run at the same moment overlap only 32–43% on cited sources. That #1 may be your average position across repeated runs — or a single point inside the noise. At minimum, confirm with the appearance rate across repeated runs of the same prompt, and ideally with a two-week-plus trend, before reporting.

We check daily and the ranking keeps changing. Is something wrong with our content?

Not necessarily. The study's key finding is that variation in simultaneous re-runs matched the day-apart range — much of the daily swing is probabilistic noise, not a change in your content or the model. Judge problems by whether the 2–4 week rolling trend has turned directionally, not by single days.

We cannot run every prompt 7 times a day. What is a realistic alternative?

Narrow the priority set. Instead of running every query seven times daily, pick a handful of queries closest to conversion for periodic repeated measurement and manage the rest on weekly or monthly trends. Using a measurement tool that automates repeated runs and period aggregation (RanketAI, Semrush AI Visibility, and others) solves the same problem. What matters is the principle — never conclude from a single measurement — more than the exact count.

Can I apply these numbers directly to my market?

Be careful with the absolute values. The study covers the Swiss market, four verticals, and Jan–Mar 2026; it is a preprint (not yet peer reviewed) and discloses the first author's affiliation with a vendor (Aurora Intelligence). The structure, however — most variation is LLM-inherent, single measurements are unreliable, period aggregation is required — holds regardless of language or market. Verify the actual run counts and windows your brand needs with repeated measurement in your own market.

Related reading

• AI Brand Recommendations: One Buyer Question Erases 62% — this article covers repeat-run noise; that one covers the reshuffle that happens when questions get more specific.
• AI Shelf Share Measurement Guide — sample sizes, relative ratios, and trend lines: the basics of measurement design.
• 27 Questions About AI Brand Visibility — an FAQ compendium on visibility measurement.