How often do AI assistants hallucinate links? We checked 48,141 URLs from three models

How often do AI assistants hallucinate links? In the 4–6 October 2026 wave, we checked the DNS status of 48,141 links from answers across 49 Ukrainian business niches: domains did not exist for 41.2% of Claude Haiku 4.5 URLs, 24.3% of Gemini 3.1 Flash-Lite URLs, and 9.5% of GPT-5 nano URLs. Local services fared worst: in veterinary clinics, 62% of linked domains did not exist. The models supplied these addresses despite a direct instruction not to invent URLs. The typical mistake pairs a correct brand name with a brand.ua address while the company actually runs on .com.ua or .com. These are hallucinated URLs in AI answers from three API models without web search, not from the consumer ChatGPT, Gemini, or Claude apps. For a brand, that is a reason to check which domain AI answers associate with its name.

How we ran the study

VYDAI collects answers to Ukrainian-language prompts about companies across 49 business categories. For this study, we used wave 11, collected from 4–6 October 2026, and checked domains on 13 October.

We tested Anthropic's claude-haiku-4-5, Google's gemini-3.1-flash-lite, and OpenAI's gpt-5-nano API models. Gemini used a preview version in wave 1; the results below focus on wave 11. Web search was off. The system prompt asked each model to name 7–12 companies, include an official website only when confident, and explicitly said not to invent URLs.

We counted a link when the answer contained a Markdown URL. For DNS checks, we treated the hostname without www as the domain. A domain counted as non-existent if a resolver returned NXDOMAIN or the domain was syntactically invalid. We checked through 1.1.1.1 and 8.8.8.8. This records DNS status on the check date; it does not identify why an address failed.

The check also looked for a working website. That broader measure includes domains without a web server, HTTP 404, 410, or 5xx responses, and sites heuristically classified as parked. Parked domains were less than 1% of links, but that classification can produce false positives. A live website also does not prove it belongs to the brand named in the answer.

How often do AI assistants hallucinate links? The results

Across the three models, the share of links pointing to non-existent domains ranged from 9.5% to 41.2%. This counts domains that do not exist at all, not broken pages. The no-working-website column is broader: it adds domains that exist but serve no usable site.

* No working website includes NXDOMAIN, domains with no web server, HTTP 404/410/5xx responses, and sites heuristically classified as parked.

** Share of answers containing at least one link, waves 3–11.

The nine-wave series from June to October stayed broadly flat. Historical tracking records 42.3% for Claude, 25.9% for Gemini, and 12.1% for GPT in wave 11. Those figures use the historical failure checks, so they differ slightly from the fresh DNS census above. Model versions stayed unchanged during this period.

Which AI model invents the most URLs?

The answer depends on whether you compare the share or the number of broken domains. Claude had the highest share: 41.2% of its links pointed to domains that did not exist. Gemini generated the largest volume of such links, about 5,400, because it linked in almost every answer and produced 22,261 URLs. Claude produced about 5,150 non-existent-domain links from 12,496 URLs. GPT produced about 1,270 from 13,384.

That is a comparison of these three models under one prompt and one test setup. It does not establish which provider is best overall. The broader differences between model recommendations are covered in our comparison of ChatGPT, Gemini, Claude, and Perplexity.

Where AI invents websites more often: 49 niches

The all-model share ranged from 2.8% in crypto to 62.0% in veterinary clinics. Local services such as vets, landscapers, and beauty salons sat near the high end, while digital businesses such as IT outsourcing and crypto sat near the low end.

The table shows the eight highest and six lowest categories by all-model share. The all-model share pools every link from the three models, so it is not the average of the three model columns. Model-level rates are shown only when the model supplied at least 100 links in that category. “Too few links” means the count was below that threshold, not that the model had no errors.

Local services reveal a difference in how the models handled uncertainty. In landscaping, Claude gave 9 links, GPT 59, and Gemini 433; in beauty salons, the counts were 30, 39, and 431; in veterinary clinics, 63, 9, and 418. Where Claude and GPT were unsure, they mostly left company names as plain text, as instructed. Gemini linked almost every company anyway, and 53–63% of its domains in these three niches did not exist.

In IT outsourcing and crypto, all three models supplied many links, from 173 to 527 per model, and failure rates were low. The gap is not simply that one model links more. The niche and the model's decision to provide a URL both affect the result. Banks had a 10.6% all-model share, including 2.0% for Gemini and 20.3% for GPT; insurance was at 32.2%, with 3.6% for Gemini and 41.8% for GPT.

Real brand, invented domain

The table contains verified examples of hallucinated web domains: an answer named a real company but supplied a domain that did not match the brand's verified address. These are the most frequent mismatches in the study, each checked against DNS and the real site's page title on 13 October 2026. Eight of the 11 share one shape: the brand name plus .ua, while the company actually runs on .com.ua or .com.

For international readers, .ua is Ukraine's country-code domain. Ukraine also has domains such as .com.ua, .in.ua, .net.ua, and .org.ua. Under Hostmaster's rules, a second-level name such as example.ua is available only to the owner of the matching trademark, while those listed third-level zones are open for registration. The pattern helps explain why a plausible brand.ua address is not necessarily the company's real site; it does not prove that the suffix caused the model's mistake. Hostmaster's domain registration FAQ describes the rules.

Wave 11 also contained 33 invalid domains with a Cyrillic letter embedded in an otherwise Latin-looking name. A Latin domain with a Cyrillic character inside cannot be the brand's address; it is a generation artifact. The link looks normal in answer text, and the mixed character becomes visible only when you inspect the actual characters.

Alfa-Bank is a different kind of mismatch. The alfabank.ua address points to an old brand name, while the verified current domain is sensebank.ua. Models can repeat a former name long after a rebrand. For steps to correct wrong brand information in an AI answer, see our guide to fixing incorrect business information in ChatGPT. How often models still recommend renamed, sanctioned, or closed services is covered in our ghost brands study.

Why does ChatGPT make up URLs?

Language models predict likely continuations from patterns in their training and the current prompt. A company name can make a familiar domain pattern seem plausible even when the model has not checked whether that address exists. In this study, the prompt explicitly said to provide an official site only when confident and not to invent URLs. The errors still appeared.

OpenAI's explanation of why language models hallucinate points to evaluation incentives that can reward guessing over admitting uncertainty. That offers one explanation for why a model may produce a plausible-looking address instead of omitting a link.

The pzu.ua and liki24.ua examples show the pattern: the model kept the recognizable brand name and fitted it into a familiar domain shape. That can look convincing in a paragraph or a list, even though a quick DNS check would have shown that the domain did not resolve on 13 October.

Does web search reduce AI hallucinations?

Web search gives an assistant a way to retrieve pages, but a retrieved URL can still be wrong, outdated, or unsupported by the answer. Google's Gemini search-grounding documentation describes how search results can ground responses. Anthropic's web search guidance also cautions that website links may occasionally not work.

For ChatGPT with search, Ahrefs' 2 September 2025 study found that cited URLs returned 404 errors at different rates: 2.38% for ChatGPT, 0.86% for Gemini, 0.87% for Perplexity, and 0.54% for Copilot. URLs from Google's top 20 had a 0.84% 404 rate. So, can ChatGPT make up links? Yes, links can still fail when search is on, but Ahrefs' page-level 404 rates are in the low single digits, far below the 9.5–41.2% domain-level non-existence rates in our no-search test. The denominators differ: a page-level 404 is not the same as a non-existent domain.

Other studies count different things. Rao, Wong, and Callison-Burch's arXiv preprint reports that 3–13% of citation URLs were likely fabricated because they had never been archived, while 5–18% did not resolve. A medical retrieval study found no URL hallucinations for GPT-4o and Gemini in its test, while noting that citations could still fail to support the claims. These results do not directly rank general-purpose assistants: the prompts, domains, and error definitions differ.

An earlier Nieman Lab report on ChatGPT's links to news investigations documented made-up article URLs even when links appeared in answers. It is a set of examples, not an error rate.

Why hallucinated web domains put brands and users at risk

A dead link costs the reader time. Fake links from AI assistants can also send a potential customer away from a brand's official site or make the company appear careless. Our study did not measure traffic loss or customer impact, so those outcomes should be treated as risks, not observed results.

There is a second risk: an address that does not exist today may be registered later. Palo Alto Networks Unit 42 published a study on 30 June 2026 covering 913 brands and 685,339 adversarial prompts. Across 2.1 million unique URLs, 37.28% pointed to unregistered domains. The researchers identified about 250,000 unique phantom domains; threat intelligence marked 0.61% of URLs as malicious, and 16.2% of those malicious URLs were phishing. They observed a gap of 18–51 days between a domain appearing in model responses and an attacker registering it.

Unit 42 used adversarial prompts designed to elicit hallucinated URLs, so those figures are not an estimate of the share in VYDAI's ordinary prompts. They do show why a non-resolving domain should not be treated as permanently harmless. A domain that becomes available can later lead to a copied page or a phishing site. Unit 42 explains the pattern in its phantom domain research.

How to verify a link generated by ChatGPT

Check the domain before sharing or relying on a link. These steps separate a spelling problem, a dead page, and a domain that belongs to a different business.

Do not rely on DNS alone. A domain may resolve and still be owned by someone else. When an answer cites a source for a factual claim, compare the cited page with the claim itself; our guide to checking sources and AI citations covers that review.

How to find which domain AI associates with your brand

Start with the prompts a customer would actually ask, such as “Which veterinary clinic in Lviv offers emergency care?” Run the same prompt on more than one model and repeat it a few weeks later. Keep the full answer, model, date, and exact URL in a log. In our study of why AI gives different answers, Claude Haiku 4.5 repeated only about 20% of the brands from its previous answer to the same prompt two weeks later, so a single response cannot show whether a domain pattern is recurring.

For a Ukrainian brand, inspect the suffix as well as the name. If your site is on .com.ua or .com, check specifically whether answers show brand.ua: that was the shape of 8 of the 11 most frequent mistakes in our table. If a repeated hallucinated URL resembles your official domain, ask your domain administrator or trademark counsel whether defensive registration or a redirect makes sense. Do not register a domain solely because one model produced it.

Consistent official-domain information across your own site and trustworthy business profiles gives readers a clear reference point. It cannot guarantee what an assistant will say. If the broader issue is that a brand does not appear in generated answers, our diagnostic guide to brand visibility in ChatGPT, Gemini, and Claude covers that separate problem.

Study limitations

This is a DNS measurement of three specific API models without web search: Claude Haiku 4.5, Gemini 3.1 Flash-Lite, and GPT-5 nano. It does not measure the consumer ChatGPT, Gemini, or Claude applications, other model versions, answers with search enabled, or every AI assistant. The figures describe one wave of Ukrainian-language company prompts, not a universal hallucination rate or a ranking of providers.

The denominator is URLs, not answers, companies, or user clicks. One answer may contain several URLs, and the models supplied very different numbers of links. DNS status was checked on 13 October 2026; domains can be registered or removed later. A domain that resolves may still be wrong for a brand, while a non-existent domain does not by itself establish why the model generated it.

The historical series uses a broader failure check than the new DNS census, which is why its wave 11 figures differ slightly. Category-level model rates are omitted when a model produced fewer than 100 URLs in that niche. We did not run an A/B test with web search on and off, and we did not measure lost visits, sales, or reputation effects. External studies cited here use different prompts, samples, and definitions, so their rates should not be compared as if they shared one denominator.

Frequently Asked Questions

Can ChatGPT make up links?

Yes. ChatGPT can return an incorrect or unavailable URL, including when web search is involved. Ahrefs found a 2.38% 404 rate among the ChatGPT URLs it checked; that page-level result is different from our DNS test of three API models without search.

What should I do with fake links from AI assistants?

Check the hostname and the page before sharing it. If you manage the brand, compare the address with the official domain and record whether the same mistake appears in later answers.

Why does ChatGPT make up URLs?

A model may produce a plausible domain pattern based on learned text patterns instead of verifying the address. Search can reduce this kind of error, but a retrieved or cited page can still be broken or fail to support the answer.

Can hallucinated domains be used for phishing?

Potentially. Unit 42 found malicious and phishing URLs in an adversarial test and documented cases where phantom domains were registered after appearing in model answers. A dead domain is not automatically a phishing site, but it may not stay dead.

How to verify a link generated by ChatGPT?

Inspect the hostname, check whether DNS resolves, open the specific page, and confirm the domain against the company's official site. If the address is unfamiliar, review its registration before entering personal or payment details.

How often do AI assistants hallucinate links when search is enabled?

There is no single rate across assistants and prompts. In Ahrefs' cited-URL sample, 404 rates ranged from 0.54% for Copilot to 2.38% for ChatGPT. Those are page-level errors, not the share of non-existent domains in our no-search test.

How we do this at VYDAI

VYDAI tracks how models mention brands across prompts, models, and dates. You can open a specific model's answer to a prompt for your brand and see the links it put into the answer text, so a recurring wrong address shows up without running prompts by hand.

For its public Ukrainian ratings, VYDAI checks a domain's DNS before attaching it to a brand, so an invented address does not end up on a company's card. There is no separate report of every invented URL in answers yet, so a suspicious link is still worth checking against the official site.

Try the English demo or create an account.