Published November 17, 2026
How a law firm tells whether AI answers really changed
Ask the same questions, in the same words, to the same four assistants, four times each, and count the runs that name the firm. A change is real when a question stays at least two runs of four above its first measurement at two measurements in a row, and a control question the firm did not touch has not moved. One run of four is noise. Nobody can promise the name, and this is general information, not legal advice.
A good screenshot is not a measurement
Ask an assistant the same question five times and the answer changes: another order, another firm, a source that was not there before. That is how these systems work, not a fault. A partner who opens ChatGPT on Monday, sees the firm named and tells the team that the pages worked has read one run. The same partner on Thursday may see the firm missing and conclude the work was undone. Both readings are noise.
A measurement is a count over repeated runs of frozen questions. What you compare is the share of runs that name the firm, before and after a change, and what you decide is whether the move is larger than the noise you see when nothing was touched.
What to freeze before the first measurement
Write down once, and then leave alone: the list of questions (up to ten, in the words a client would type, without the firm’s name), the country and language of each, the four assistants, the number of runs per question, which is four, and the date. Save each full answer, not a screenshot of its first lines, because the name may sit in the third paragraph or only in the sources.
Add one control question: a matter the firm handles but on which it will publish nothing during the period. The control tells you whether the assistants themselves moved. If it goes from zero to three of four with no page changed, a model update did it, and the firm should not take the credit.
Changing one word in a question, even to make it sound better, starts a new question with a new first measurement. People do this to get a better number, and it is the quickest way to make a series worthless.
The reading rule
A question counts as changed when, on the same assistant, it stays at least two runs of four above its first measurement at two measurements in a row. One run of four is noise, and so is a single measurement. If the control moved as well, the change is not credited to the firm.
Then read where the name came from. If the assistant names the firm and cites the firm’s own page, the page did the work. If it names the firm from a directory that already listed it, nothing you published moved it. The cited source is the evidence; the name alone is not.
Do not average the four assistants into one score. They do not behave as one, and a figure such as 50% hides that two of them moved and two did not. Keep one line per assistant per question.
A worked example, step by step
Example: a hypothetical firm, not a sayurname client. Mersey & Hale is an invented five-lawyer employment firm in Manchester that advises small employers. Its settlement-agreement note for the employer is £450 to £700. The old site had one “Employment” page listing twenty topics. The measured question was “Which solicitors in Manchester help a small employer with a settlement agreement?”.
The firm froze six questions: five on employment and one control, about commercial leases in Leeds, where it would publish nothing. Each was asked four times on each of the four assistants, which makes 96 answers at every measurement.
- Froze six questions, four assistants and four runs each, and saved every full answer with its date.
- Week 0: read all 96 answers. The firm was named in none of the 16 runs on the settlement question, and in none on the control.
- Replaced the twenty-topic page with one settlement-agreement page: the solicitor’s name, where she is admitted, a link to the regulator’s lookup, the £450 to £700 range, and what the note does not cover.
- Week 4: ChatGPT named the firm in 1 run of 4 and the others in none. One run of four, so nothing was credited.
- Week 8: ChatGPT 3 of 4 and Perplexity 2 of 4, Gemini 1 of 4, Claude 0 of 4. The control was still 0 of 4 on every assistant.
- Week 12: ChatGPT 3 of 4 and Perplexity 3 of 4 again, so both were counted as changed. Gemini and Claude stayed unreadable, and the page was left alone for another four weeks.
Before filing the result, the firm read the sources. Perplexity cited the new page in every run that named the firm. ChatGPT cited the page in two of its three naming runs and a legal directory in the third, so the firm credited the page for most of that move and not all of it.
It did not report “50% better”. It reported that two assistants moved, two did not, and what it would change next for the two that did not. Another firm with the same page might see Gemini move first, or nothing at all in twelve weeks.
When not to measure again
Do not re-measure a week after a page changed. Assistants and the sources they read refresh at different speeds. In the example the first move took four to eight weeks, and some firms see none in twelve. A daily reading draws a chart of noise and invites a decision to rewrite the page every time it dips.
Nor should a firm start a series for a question it would never act on. If the firm will not write the page, the measurement only records that it is absent, week after week. And do not widen the list while a series is running: new questions get their own first measurement, apart from the old ones.
How the work runs
It starts with the free test. Once subscribed, we measure up to 10 questions every week and publish as much as we judge necessary for AI to cite your firm in the answers you want. Volume and format follow what each question needs, and that part is our call. We also ask for inclusion in the sites the assistants use. What we promise is the request, not the listing.
It costs $299 a month, with no lock-in. A one-time $599 bonus is charged only when an AI starts naming your firm for a measured question, checked by a person on our team, or Google Analytics shows a visit coming from an AI. Your firm stays responsible for validating the legal content of what is published, and each piece comes with one revision. It is not legal advice, and it promises no client.
Questions
How many runs are enough?
Four per assistant per question is the minimum in this method. It separates none from most of the runs. It does not separate 40% from 50%, and the firm should not try.
Can we ask the assistants ourselves from the office?
You can, but your history, your location and earlier chats can change the answer. Use a clean session, the same setup every time, and save the full answer.
What if the control question moves?
Then the assistants moved, not the firm’s pages, and the change is not credited. Keep the series going and read it again at the next measurement.
How often should a firm measure?
By hand, once a month is enough. A weekly measurement, which the subscription does, gives four readings a month, so the rule of two in a row can be applied sooner.
Does a changed answer mean new clients?
No. It means the assistant now names the firm for that question. Whether someone then calls depends on the page, the lookup and the matter.