Late on a weeknight, a runner with a swollen knee types a simple request into a chatbot: find a good orthopedic surgeon nearby. AI doctor recommendations like the one that comes back tend to look alike. There are three practices, each with a short, confident reason, written in the calm voice of a well-informed friend. Nothing on the screen says “sponsored,” and the answer doesn’t need that label to persuade.
This is no longer unusual. In a nationally representative US poll of 1,343 adults taken in February and March 2026, about one in three said they had used an AI chatbot for health information in the past year, and four in ten of those users said a major reason was to look things up before deciding whether to see a provider [KFF, AI Chatbots for Health Information Poll, 2026]. For a growing share of patients, the chatbot now works like the front desk of medicine. An industry has noticed, and it is selling ways to influence who gets mentioned there.
Nobody is buying these recommendations outright. In personal health conversations on the largest chatbot, no one pays the AI company for placement. The problem is that search advertising has always carried a label and the AI answer carries none, even when the material behind it was written specifically to win that answer. Labels never stopped every kind of marketing influence, but they were the only safeguard patients could check for themselves. That leaves two questions: what shapes the answer, and who is responsible for telling patients?
Why AI Doctor Recommendations Feel Like Advice, Not Ads
On a search results page, the market is visible. Paid listings carry a tag, organic links sit below them, and the patient does the comparing. An AI answer squeezes all of that into one paragraph. The comparing has already been done somewhere the patient can’t see, and what’s left reads like a verdict.
People behave differently as a result. In a Pew Research Center analysis of 68,879 real Google searches by 900 US adults in March 2025, people clicked through to a website in 8% of visits when an AI summary appeared, compared with 15% when none did, about half as often [Chapekis, AI Summary Click Behavior, 2025]. They clicked the sources cited inside the summary in just 1% of visits. It’s worth adding that most searches never led to a click on a result, with or without a summary. These were also general searches rather than health searches, so the pattern is a strong hint, not a measured fact about medical decisions.
Even so, the direction is clear enough: when the answer arrives already assembled, fewer people check where it came from.
None of this makes AI recommendations inherently worse than the old way of finding a doctor. Some researchers argue that AI could help patients compare providers on experience and outcomes rather than on who markets best, by gathering credentials and track records that are scattered across many websites [Weeks, AI Promise and Peril in Healthcare, 2024]. That promise hasn’t been tested yet, but it’s a real one, and it’s why the stakes are high. A tool that could steer patients toward quality could just as easily steer them toward whoever is most visible. Which way it goes depends less on the model itself than on the material it reads.
What the Machine Reads Before It Recommends
Search-enabled AI tools build their answers from web pages they pull in at the moment a question is asked, so what is published online shapes what they say [Aggarwal, GEO, 2024]. Practice websites, directory profiles, clusters of reviews and blog posts about knee replacement recovery all become raw material. The AI never visits the clinic; it only reads about it.
Rewrite that material and you can nudge the answer. A 2024 conference paper named the practice generative engine optimization, or GEO. In experiments outside medicine, the researchers found that rewriting web content by adding citations, statistics and clearer phrasing could make a source as much as 40 percent more visible in AI-generated answers, though the effect varied by topic and query type [Aggarwal, GEO, 2024]. A separate 2024 preprint used a catalog of fictitious coffee machines and showed that inserting a carefully engineered text sequence into a product page could make a rarely mentioned product far more likely to appear as the top recommendation [Kumar, Manipulating LLMs for Product Visibility, 2024].
Neither study tested doctors or hospitals, and no study has yet. Not all GEO is manipulation, either. Much of it just makes accurate information easier for a machine to read, which could improve answers rather than distort them. What the evidence supports is narrower, though still unsettling: outside medicine, AI recommendations have been shown to shift depending on how sources are written, and nothing so far suggests medicine is immune. If the pages are shaped by marketing, the answer may be too.
The Ad Slot Closed. The Answer Did Not.
For now, health is mostly shut out of the formal advertising channel inside chatbots. OpenAI began testing ads in ChatGPT in the US on February 9, 2026, placing labeled, sponsored units below answers and stating that ads do not influence what the model says [OpenAI, Ads in ChatGPT, 2026]. Its policy keeps ads away from sensitive topics, including personal health conversations. Some health advertisers may qualify under strict criteria, but their ads still can’t appear next to a person’s own medical questions. The guardrail is reasonable, but it has a side effect.
With the labeled slot closed, influence moves to places that carry no label. US marketing agencies now openly sell services promising to get medical practices recommended when patients ask ChatGPT or Gemini for a doctor. One recently announced agency describes cleaning up how a practice appears across physician directories and booking sites so that AI systems recommend it more consistently. Another sells “get recommended” packages for practices in dozens of US cities, and its sales pitch rests on the claim that patients treat an AI suggestion much like personal advice. These services are advertised openly. What the patient doesn’t see is the result: they get the answer, never the campaign behind it.
| Channel | Marked as paid? | Who is paid | What the patient sees |
|---|---|---|---|
| Search engine ad | Yes, labeled “sponsored” | The search engine | A tagged listing among other results |
| Booking marketplace listing | Varies by platform | The booking platform (per booking, in Zocdoc’s case) | A list of available doctors |
| Chatbot sponsored unit | Yes, labeled, and excluded near personal health chats | The AI company | Nothing, in a personal health conversation |
| AI answer shaped by optimized content | No | A marketing agency, not the AI company | A neutral-sounding recommendation |
This article is concerned with the last row, which existing advertising rules weren’t written to cover. US regulators have hinted at the problem. Federal Trade Commission staff wrote in 2023 that generative AI output should clearly separate what is paid from what is organic, and that people should know when a commercial relationship is steering them toward a particular service provider [Atleson, The Luring Test, 2023]. That guidance addresses a commercial relationship between the AI product and the provider. It says nothing explicit about an answer shaped upstream, at the level of the sources, by parties the AI company has never dealt with.
A chatbot company can decline to sell ad space next to an answer, but it has no easy way to stop its model from reading pages written to be picked.
Legal Is Not the Same as Visible
US law already has precise terms for paying to steer patients, and they’re useful here. The federal Anti-Kickback Statute bars paying for referrals of patients covered by federal health programs such as Medicare and Medicaid. Its clearest protection for referral services, a regulatory safe harbor, covers fees that are uniform, cost-based and do not rise with the number of patients sent; arrangements outside it are judged case by case [Barnes & Thornburg, OIG Pay-to-Play Referral Opinion, 2026]. The reasoning is long-standing: when payment rises with volume, referrals tend to follow the payment.
The line is a fine one, as two cases show. When the booking platform Zocdoc moved in 2018 to charging practices per new-patient booking, it initially kept Medicare and Medicaid patients off the feature until federal regulators issued a favorable opinion in 2019, an opinion that applies only to that company’s specific setup [Fader, Zocdoc Anti-Kickback Approval, 2019].
In June 2026, regulators reached the opposite conclusion on a different model. In Advisory Opinion 26-15, they declined to approve a referral platform where home-health agencies that paid got a faster lane to hospital referrals, warning that patients could end up steered by who paid rather than by quality of care [Office of Inspector General, Advisory Opinion 26-15, 2026]. That case had nothing to do with AI, but its reasoning maps closely onto the AI question.
The gap shows up with flat fees. A practice paying a flat monthly fee to an agency that rewrites its web presence would most likely be treated as buying marketing rather than referrals, though no regulator has addressed the question. Under the FTC’s updated endorsement rules, adopted in June 2023, a paid review or testimonial must disclose the relationship clearly, and the rules reach the agencies and reputation managers in the middle [Keller and Heckman, FTC Updated Endorsement Guides, 2023].
What those rules don’t yet say is what happens when an AI quietly blends that content into an answer. No US rule identified for this article specifically addresses paid influence on what an AI says about medical providers, and none of these arrangements has been tested in court.
As far as current rules go, then, this kind of influence appears to be legal. It simply reaches patients without a label.
When the Recommendation Becomes the Booking
Until recently, an AI recommendation still had to pass through a human hand. The patient read the answer, perhaps checked a second source, then called the office. AI agents remove some of those steps. Meta’s Muse, launched in the US in September 2026, can fill out forms, book and pay on a user’s behalf, and says it asks permission before sensitive steps such as sending an email or making a purchase [Meta, Introducing Muse, 2026]. Medical booking isn’t one of its featured uses, though it’s an obvious one.
Muse already checks out through a payments partner. If agents start booking clinics, the next question is who pays whom when they do. Meta says Muse keeps conversations away from its ad systems; one tech outlet noted that, for now, this is a policy promise rather than something built into the system [Constantin, Meta Launches Muse, 2026]. A policy promise may well be sincere, but a company can change it later.
Human psychology adds to the risk. People tend to over-trust automated advice, a well-documented pattern called automation bias. A systematic review of 74 studies found it was shaped by trust, workload and time pressure, and softened when systems presented information rather than bare recommendations [Goddard, Automation Bias Systematic Review, 2012]. Most of that research involved clinicians, not consumers. It is reasonable to expect the tendency to matter more when an agent can act on its own suggestion, though this has not yet been studied for AI booking agents. An approval button only protects patients if the person pressing it still feels free to say no.
Some situations shouldn’t involve a chatbot at all. For warning signs such as chest pain, shortness of breath, a severe allergic reaction, or confusion, the right move is to skip the chatbot and seek care directly [Mount Sinai, Blind Spots in AI Medical Triage, 2026]. That holds no matter how trustworthy the recommendation engine is.
A well-designed agent could get patients from a symptom to the right doctor faster. A poorly designed one could just as quickly turn a practice’s marketing budget into appointments on a patient’s calendar.
Key Takeaways
- AI chatbots are becoming a common first stop for health decisions, with about a third of US adults using them for health information in the past year.
- When an AI summary appears, people click through to sources about half as often, which puts more weight on whatever shaped the answer.
- Experiments outside medicine show that rewriting web content can measurably change what AI tools recommend; no study has yet tested this for doctors.
- A visible market now sells medical practices on being recommended by chatbots, while health ad slots inside chatbots remain largely closed.
- US referral law targets volume-based payments; flat-fee marketing that shapes AI answers appears to fall outside it and carries no label.
- Asking an AI why it chose a particular doctor is a simple first test of whether the recommendation deserves trust.
FAQ
Are AI doctor recommendations paid advertisements?
Usually not in the direct sense. ChatGPT, the largest chatbot, keeps ads out of its answers and away from personal health conversations [OpenAI, Ads in ChatGPT, 2026]. The influence works upstream: agencies optimize the web content AI tools read, which can shift what the answer says without any label.
Can a medical practice pay to be recommended by ChatGPT?
It depends. A practice cannot buy a spot inside ChatGPT’s answer itself, but it can pay agencies to rewrite its online presence so AI tools are more likely to cite it. Experiments outside medicine show such rewriting can raise visibility in AI answers by as much as 40 percent [Aggarwal, GEO, 2024].
Is it legal to steer patients with AI recommendations in the US?
It depends. Federal law bars volume-based payments for referrals of Medicare and Medicaid patients, and regulators rejected a pay-for-priority referral platform in June 2026 [Office of Inspector General, Advisory Opinion 26-15, 2026]. Flat-fee marketing that shapes AI answers has not been tested against these rules.
Should an AI chatbot be used to find a doctor for urgent symptoms?
No. For chest pain, shortness of breath, a severe allergic reaction or confusion, researchers advise seeking care directly rather than relying on chatbot guidance [Mount Sinai, Blind Spots in AI Medical Triage, 2026]. AI tools are better suited to routine searches, where there is time to check a second source.
References
- KFF. Poll: 1 in 3 adults are turning to AI chatbots for health information, equaling the share who use social media for health [Internet]. San Francisco (CA): KFF; 2026 Mar 25.
- Chapekis A, Lieb A. Google users are less likely to click on links when an AI summary appears in the results [Internet]. Washington (DC): Pew Research Center; 2025 Jul 22.
- Weeks WB, Lavista Ferres JM, Weinstein JN. Artificial intelligence: promise and peril in achieving the quadruple aim in healthcare. Front Artif Intell. 2024;7:1430756.
- Aggarwal P, Murahari V, Rajpurohit T, Kalyan A, Narasimhan K, Deshpande A. GEO: generative engine optimization. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2024.
- Kumar A, Lakkaraju H. Manipulating large language models to increase product visibility [Preprint]. arXiv:2404.07981. 2024.
- OpenAI. Ads in ChatGPT [Internet]. San Francisco (CA): OpenAI; 2026 [cited 2026 Oct 5].
- Atleson M. The Luring Test: AI and the engineering of consumer trust [Internet]. Washington (DC): Federal Trade Commission; 2023 May 1.
- Barnes & Thornburg LLP. HHS-OIG issues unfavorable advisory opinion on pay-to-play referral software platform arrangement [Internet]. 2026 Jul 9.
- Fader ED. Zocdoc fees get anti-kickback approval [Internet]. Rivkin Radler LLP; 2019 Sep 18.
- Office of Inspector General. Advisory Opinion 26-15 [Internet]. Washington (DC): US Department of Health and Human Services; 2026 Jun 30.
- Keller and Heckman LLP. FTC publishes updated endorsement guides [Internet]. Washington (DC); 2023 Jul 18.
- Meta. Introducing Muse: the world’s first personal AI agent built for everyone [Internet]. Menlo Park (CA): Meta; 2026 Sep 8.
- Constantin AM. Meta launches Muse, a personal AI agent that books, buys and negotiates for you [Internet]. The Next Web; 2026 Sep 9.
- Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121-7.
- Mount Sinai Health System. Research identifies blind spots in AI medical triage [Internet]. New York (NY): Mount Sinai; 2026 Feb 24.
Joonpyo Hong, MD is a board-certified otolaryngologist practicing in Korea. This article reflects his clinical interpretation of published research and does not constitute individual medical advice. This article is not intended to advertise or promote any specific company or product.
For more articles:
https://curiousmd.com/ai-ent-referral/
https://curiousmd.com/chatgpt-medical-triage-ent-review/
Link out to:
KFF poll on how many US adults use AI chatbots for health information: https://www.kff.org/health-information-trust/poll-1-in-3-adults-are-turning-to-ai-chatbots-for-health-information-equaling-the-share-who-use-social-media-for-health/
Pew Research Center on how AI summaries change clicking behavior: https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/
OpenAI’s policy on ads in ChatGPT and sensitive health topics: https://help.openai.com/en/articles/20001047-ads-in-chatgpt
FTC staff guidance on separating paid from organic AI output: https://www.ftc.gov/business-guidance/blog/2023/05/luring-test-ai-engineering-consumer-trust
HHS Office of Inspector General, Advisory Opinion 26-15: https://oig.hhs.gov/compliance/advisory-opinions/26-15/
Meta’s announcement of the Muse personal AI agent: https://about.fb.com/news/2026/09/introducing-muse-personal-ai-agent/