LLM tracking (also called AI rank tracking) follows over time whether AI systems name a business for fixed customer questions. Because answers vary a lot, shares across many questions and repeated runs count rather than single positions. The key metrics are mention rate, share among competitors, position in the answer and your own website as a source. Good tracking stores the original answers and measures on a fixed schedule.
What does LLM tracking mean?
The term has two meanings that are worth keeping apart:
- In software development, LLM tracking means logging requests to language models: costs, response times, errors. Tools such as MLflow or Langfuse exist for that. That is not what this guide is about.
- In marketing, LLM tracking means following your own visibility in AI answers. Other names for it are AI rank tracking, AI visibility tracking or AI monitoring.
This guide covers the second meaning: how to track over weeks and months whether ChatGPT, Gemini, Perplexity and Google AI Overview name your business for your customers’ questions.
Why AI answers have no fixed rankings
A Google results list is largely the same for all users in the same place. AI answers, by contrast, are generated fresh every time. They depend on the exact wording, the time, the sources found at that moment and the model. Our own measurement shows how much this varies:
In our web agency study, the same AI named only 35 percent of the agencies again one week later for the same question. 86 percent of the agencies were recommended by only one of the three AIs. Two slightly differently worded questions produced only about 4 percent overlap in the same AI.
The metrics that really count
| Metric | Question | Example |
|---|---|---|
| Mention rate | For how many questions am I named? | 4 of 20 questions = 20% |
| Mention rate per AI system | Where am I strong, where am I missing? | ChatGPT 30%, Gemini 10% |
| Share among competitors | How often compared with competitors? | 18% of all mentions |
| Position in the answer | Am I named first or last? | second on average |
| Own website as a source | Does the AI link to my pages? | in 3 of 20 answers |
| Questions without a mention | Where does the AI name others but not me? | 12 questions with competitors |
One rule for honest numbers: count questions that contain your company name separately. Anyone asking “What does company X offer?” almost always gets company X named. Such questions inflate the mention rate. How to calculate your share among competitors is shown by the calculator in our share of voice guide.
Setting up LLM tracking in five steps
- Choose questions. 10 to 30 questions real customers ask, without your company name. Good sources are customer conversations, email enquiries and the queries in Search Console. Our AI prompt generator helps.
- Fix systems and market. Which AI systems, which language, which country, which device? These conditions then stay the same.
- Set a schedule. Weekly is a good middle ground for most businesses. Daily measurements add cost and noise, monthly ones miss changes.
- Store the original answers. Not just “named yes/no”, but the full text and the sources. Only then can you explain later why something changed.
- Define competitors. Three to ten real competitors whose mentions you count as well. That way you see your share instead of an isolated number.
A detailed template with a measurement log is in our guide to measuring AI visibility.
Reading the results correctly
Raw data only becomes useful when you ask the right questions:
The mention rate is rising – success?
First check whether the question set has changed. A rise is only comparable with the same questions, the same systems and the same market.
One AI names me, the other never does?
Look at the sources. In our study Perplexity often relied on directories, ChatGPT more on websites. Missing listings explain many gaps.
A competitor suddenly appears everywhere?
Check which new sources name them: an article, a comparison portal, a new directory. That is often where your opportunity lies too.
The numbers jump from week to week?
That is normal. Look at the rolling average over four weeks rather than single measurements.
From the gaps you derive concrete tasks: a missing service page, a missing directory listing, a page that does not answer a customer question directly. How to prioritise such tasks is covered in our GEO audit guide.
Looking at LLM tracking and AI traffic together
LLM tracking shows whether you are named. Your web analytics shows whether that turns into visits. In the GA4 traffic acquisition report you can analyse referrals from chatgpt.com, perplexity.ai or gemini.google.com. The step-by-step guide is in measuring AI traffic.
Together the two give a complete picture: if the mention rate rises but AI traffic does not, you are named but not linked. If AI traffic rises without more mentions, a single strong page is probably doing the work. For the monthly report, use the template in our GEO reporting guide.
By hand or with a tool?
A spreadsheet is enough to start: 10 questions, four systems, once a week. That is 40 answers to read, note and compare. After a few weeks it gets tedious, and small changes get lost.
Nalqerio does this automatically: your customer questions are measured in ChatGPT, Gemini, Perplexity and Google AI Overview, the original answers are stored and mentions, competitors and sources are evaluated over time. If you want, you get a notification when you lose mentions. Questions containing your company name are counted separately in the metrics. The free AI check gives a first impression for ChatGPT.
Frequently asked questions
What is LLM tracking?
In marketing, LLM tracking means following your own visibility in AI answers over time: whether and how often ChatGPT, Gemini, Perplexity and others name a business for fixed customer questions. In software development the term instead means logging costs and requests.
What is an AI rank tracker?
A tool that regularly asks AI systems customer questions and checks whether and where a brand is named. Because AI answers vary, good trackers show shares across many questions instead of single positions.
How often should I measure my AI visibility?
Weekly is a good schedule for most businesses. Evaluate the average over several weeks, because single measurements vary a lot.
Why do I get different answers to the same question?
AI answers are generated fresh every time and depend on wording, timing, the sources found and the model. In our study the same AI named only 35 percent of the same providers again one week later.
How many questions do I need for meaningful tracking?
For small businesses 10 to 20 well-chosen customer questions are enough. More important than the number is that they reflect real buying intent and stay unchanged.
Sources and further reading
- Nalqerio study: Which web agencies do ChatGPT, Gemini and Perplexity recommend?
- Google Analytics Help: Traffic acquisition report
- OpenAI Help Center: Searching the web with ChatGPT
- Nalqerio: measurement methodology
Methods, product scope and sources reflect the publication date shown. Examples are not measurements of your website. Linked product pages provide current plan and provider details.
