Research

How noisy are AI recommendations?

We asked AI the same local questions 238 times over 8 weeks. Here’s how much the answers move.

What we looked at

Every market-week from 2026-W33 to 2026-W40 (8 complete weeks) with at least 10 web-search answers: 12 market-weeks across 6 markets, 238 answers from Perplexity, ChatGPT and Claude. 28 market-weeks had fewer answers and are left out, as they are from every trend we report. No market or business is left out for how its numbers look. Refusals and empty answers aren’t counted. Computed October 11, 2026; it updates every Monday.

Why one check misleads

A single check is one draw from a wide spread. That’s why every number we report pools many answers and shows how many.

How wide the ranges are

For a typical business near the top of its market, named in about 43% of answers (the median for businesses ranked in the top 5 of their market-week, n = 60), the 95% range at different numbers of answers:

Answers95% range
10a quick check17%–69%
24a few days25%–61%
72a full week32%–55%
144a full week, deep sampling35%–51%

Doubling the answers narrows the range by about 30%, not by half. That’s why deep sampling helps, and why a trend pools several weeks.

How this was worked out

More on the method: how Algotally measures.

Numbers that say how sure they are.

Every rate Algotally reports comes with its answer count and a 95% range.