Stop asking what your position is in ChatGPT
It is the first question every team trained on SEO asks about AI: "where do we rank?" It is also the wrong question, and not by a little. AI answers are non-deterministic — the same prompt produces different responses each time — so there is no position to hold. There is only a probability of showing up.
the short answer
AI responses are non-deterministic by design: a model generates text by sampling probabilistically, so the same prompt gives different answers each run. There is no rank and no position one — only whether your brand appears, which varies by run, user and moment. So the metric is not a position but a probability of appearing, measured by running prompts many times and recording the share of responses you show up in. Stability lives in the rate across many samples, never in a single answer.
key takeaways
- LLMs are non-deterministic by design: they generate text by probabilistic sampling, so the same prompt gives different answers each time.
- There is no "position one" or stable ranking in AI — your brand is either in a given answer or not, and that varies by run, user and moment.
- The right metric is not a position but a probability of appearing: run the same prompt many times and measure the share of responses you show up in.
- Sources of variation: sampling/temperature, personalisation (history, context, location), model version, and live retrieval.
- Brands with strong, consistent signals (clear entity, distributed authority) appear in a higher share of responses even as the wording varies — signal consistency raises the probability.
the same prompt, run ten times (illustrative)
In words, so the dots do not carry it alone: imagine running one representative prompt ten times. Your brand shows up in six of the answers and is missing from four. Read as ten separate facts, that looks like chaos — present, absent, present, absent. Read correctly, it is a single fact: an appearance rate of roughly sixty percent for this prompt. No run is predictable; the rate is. The numbers here are illustrative, not from a study — the point is the shape of the thing, which is a probability, not a rank.
Why this is not a technicality
It would be easy to file all this under pedantry — rank, probability, who cares, just tell me if I show up. But the distinction changes what you do, not just how you talk. If you believe there is a rank, you will read a single bad answer as a drop and panic, or a single good one as a win and relax, when both were just samples that could have landed the other way. You will buy a tool that promises a tidy "AI position" and trust a number that does not mean what it claims. You will optimise toward moving up a ladder that is not there, and misattribute the random wobble of sampling to whatever you changed last week. Mistaking a probability for a position is not a vocabulary error; it is a measurement error that leads to acting on noise.
Get the model right and the opposite happens. You stop reacting to individual answers and start watching a rate that moves slowly and for real reasons. You judge a change by whether your appearance probability rose across many runs, not by whether one screenshot looked good. You compare yourself to competitors on a like-for-like basis — who appears more often for the questions that matter — instead of trading anecdotes. The honesty of the metric is what makes it useful: because it admits the channel is probabilistic, it measures the thing that is actually stable underneath the variation, and that is the thing worth managing.
The non-determinism problem, in three parts
Why AI answers vary, why "ranking position" has no meaning here, and what to measure instead. Open each layer for the part that changes how you track visibility.
01 Why AI is non-deterministic
A search engine, asked the same query twice, returns essentially the same ranked list — it is looking up an index. A language model does something different: it generates an answer one token at a time, and at each step it samples from a probability distribution over possible next words. That sampling is what makes the output fluent and varied, and it is also what makes it unpredictable: run the same prompt again and the model may roll the dice differently, producing a different phrasing, a different set of examples, a different selection of brands. This is not a malfunction to be fixed; it is how the technology works. On top of the sampling sit further sources of variation — personalisation from your history and context, the particular model version answering, and live retrieval that pulls whatever sources are available at that moment. The result is a system whose individual outputs are genuinely not repeatable.
02 Why a "ranking position in AI" is a fiction
The instinct carried over from SEO is to ask "where do I rank," and in AI that question has no answer, because the structure that gives it meaning does not exist. There is no results page with a first slot and a tenth; there is a generated answer in which your brand is either mentioned or not. Across two runs of the same prompt you might appear in one and vanish in the next, not because you moved from position three to position eight, but because there are no positions — only presence or absence, sampled fresh each time. Any tool or report that hands you a single "AI rank" is therefore describing something the system does not produce; it has taken one sample and dressed it up as a standing. Letting go of the ranking metaphor is the first real step, because as long as you are looking for a position you will misread the noise of variation as movement up and down a ladder that is not there.
03 What to measure instead: appearance probability
If there is no position, what is there? A probability — the rate at which your brand appears across many runs of a representative prompt. That number is real and well-behaved even though any single answer is not. Run a prompt once and you learn almost nothing; run it twenty times and the share of responses naming you stabilises into something you can compare against a competitor and track over months. This reframes the whole measurement task: you are not checking a rank, you are estimating a rate, the way you would estimate how often a weighted coin lands heads. Vary the phrasing too, because real users ask the same thing many ways, and aggregate across engines, because each behaves differently. What you end up with is not a tidy position but an honest distribution — appears seventy percent of the time here, thirty there — which is a truer picture of your visibility precisely because it matches how the channel actually behaves.
Signal versus noise: what actually moves the rate
If individual answers are random, what makes a brand appear in seventy percent of them rather than ten? The answer is signal strength, and it is the encouraging part of an otherwise slippery picture. The variation between runs is noise layered on top of a real underlying tendency: how strongly the model associates your brand with the query. A brand the model knows clearly — unambiguous identity, consistent description across the web, recognised authority in the space — surfaces in a high share of responses even though the exact wording changes each time. A brand the model is unsure about appears rarely and erratically, because when a model is uncertain it tends to leave a name out rather than risk being wrong. So the rate is not arbitrary; it is a readout of how well-established you are in the model’s sense of the category.
This is what makes the probabilistic view actionable rather than fatalistic. You cannot control any single answer, but you can raise the floor the answers are sampled from — by making your entity unmistakable, your description consistent everywhere the model reads, and your authority real enough that the association is strong. Do that and your appearance rate climbs, not because you gamed a position but because you became, on average, a more likely answer. The noise stays; the signal underneath it rises. Measuring the rate honestly is what lets you see that rise at all, which is the practical reason to insist on the probability and refuse the fictional rank.
Share of voice: the rate becomes a comparison
An appearance rate on its own — "we show up in sixty percent of responses to this prompt" — is informative, but it becomes far more useful the moment you measure your competitors the same way. Run the same prompts the same number of times and record how often each brand in the category appears, and you have a share of voice: not just how visible you are in absolute terms, but how visible you are relative to the names you actually compete with for that intent. That comparison is where the metric earns its keep. An appearance rate that drifts from sixty to fifty-five percent might be noise; an appearance rate where a rival climbs from twenty to forty while yours holds is a real competitive shift worth understanding. Because every brand is being sampled from the same non-deterministic system, the relative picture is steadier and more honest than any single brand’s number read alone — the noise affects everyone, so comparing within the same runs cancels much of it out.
A simple sampling protocol
None of this requires sophistication, just discipline, and the protocol fits in a paragraph. Start by choosing a set of prompts that represent how real buyers ask about your category — several phrasings of each genuine intent, not one tidy keyword, because users vary the wording and you want to capture that spread. Run each prompt several times on each engine that matters to you, since the engines diverge and a single run of a single phrasing tells you almost nothing. Record, for every run, whether your brand and each competitor appeared, and roll those up into appearance rates and a share of voice. Then repeat the whole thing on a fixed cadence — monthly is usually enough to see real movement without drowning in run-to-run noise — so you are comparing like with like over time.
Avoid the two opposite failure modes: under-sampling, where a handful of runs lets randomness fake a trend, and false precision, where a rate reported to the decimal pretends to be a fixed measurement rather than an estimate. Sample enough that the rate is stable, report it as the approximate figure it is, and act on changes large enough to clear the noise — and you have what most teams chasing an "AI rank" never get: a measurement that corresponds to how the channel behaves.
No ranking in AI: quick answers
So is measuring AI visibility pointless?
No — it is just probabilistic rather than positional, and confusing the two is what makes people give up too early. You cannot get a stable "rank," because there is no rank; but you can get a stable rate. If you run a representative prompt enough times, the share of responses your brand appears in settles into a meaningful number, the way a coin’s heads-rate settles around fifty percent even though no single flip is predictable. That appearance rate is real, comparable to competitors, and trackable over time. What is pointless is treating a single AI answer as a verdict — one response is one sample from a distribution, not a measurement. The discipline is to stop reading individual answers as scores and start reading the distribution across many answers as the metric. Measured that way, AI visibility is not pointless at all; it is just measured like a probability, which is what it is.
How many times should I run a prompt?
Enough that the number stops moving much when you add more runs — which in practice means more than a couple and fewer than you fear. A single run tells you almost nothing, because any one answer could have gone either way. A handful starts to show a pattern. By a dozen or more runs of the same prompt, the appearance rate usually stabilises enough to compare honestly against a competitor or against last month. The exact count depends on how variable the prompt is and how precise you need to be, but the principle is simple: you are estimating a rate, and estimates get more reliable with more samples, with diminishing returns after a point. Just as important is varying the prompts themselves — real users phrase the same intent many ways — so a reliable picture comes from several phrasings each run several times, not one phrasing run a hundred times.
Can I get a stable number at all?
Yes, at the level of a rate, even though you cannot at the level of a single answer — and that distinction is the whole point. Individual responses are genuinely unpredictable: the model samples, personalises, and pulls live sources, so any one reply can surprise you. But aggregate behaviour is far steadier. Across many runs, a brand that the model strongly associates with a query appears most of the time, a weak one appears rarely, and the in-between cases land in between — and those rates hold up well enough to act on. So the stable number you want is not "position three," it is "appears in roughly seventy percent of responses to this kind of question." That figure moves slowly and meaningfully as your underlying signals change, which makes it a far better thing to manage toward than a position that does not exist. Stability lives in the aggregate, not the instance.
Does this mean tools claiming an "AI rank" are wrong?
A tool that reports a single fixed "rank in AI" is selling a number the underlying system does not produce, and you should be sceptical of it. There is no position one in a generative answer the way there is in a list of search results; there is only how often you appear and how you are characterised when you do. A good tool embraces that — it samples prompts repeatedly, across engines, and reports appearance rates, share of voice, and sentiment as distributions, with some honesty about variance. A weak tool flattens all that into a tidy rank because a rank is easier to sell and easier to put in a dashboard, even though it misrepresents how the channel works. The test is simple: ask whether the number comes from many runs treated as samples, or from one run treated as truth. The first is measurement; the second is a screenshot dressed up as a metric.
A note on sources and certainty
This is written in early 2025, and it rests on how language models work rather than on any single study, which is deliberate: the non-determinism described here is a property of probabilistic text generation, not a finding that might be revised. We have kept the worked example explicitly illustrative — six appearances in ten runs is a teaching number, not a measurement of any brand — because the honest claim is about the shape of the metric, not a specific figure. Rigorous quantification of how much AI answers vary, across engines and categories, was still being assembled as we wrote, and we have not borrowed later numbers backward to lend false precision. What is durable is the principle: AI visibility is a probability estimated by sampling, not a position to be ranked, and the brands that measure it that way will read their progress correctly while the rest chase a number that does not exist. The AC Group has spent 27 years preferring an honest, uncertain measurement to a confident, meaningless one.