At 11 p.m., when a hearing aid stops pairing and the manual is nowhere to be found, the thing within reach is a chatbot, not an audiology clinic. Three audiologists decided to check what people actually get when they do that. They collected 44 common hearing aid questions, ran each one through ChatGPT and Gemini, and graded every answer against the official manufacturer manuals. So called, AI hearing aid advice.
Both models came back with a perfect median score for clarity. On medical and technical accuracy, they came apart.
The study appeared in August 2026 and is one of the few direct head-to-head evaluations of something millions of people already do. Here is what the numbers say, including the parts that favor AI, with the verdict left to you.
What the audiologists actually tested
The 44 user queries were sorted into seven categories covering everyday device management. Three expert audiologists scored each response for comprehensibility and medical accuracy, with official manufacturer manuals as the gold standard. Agreement between raters was strong in both domains (ICC = 0.74, p < 0.001), so these are not one person’s impressions [Batuk, ChatGPT and Gemini on hearing aid management, 2026]. Repeatability was measured separately, by algorithm rather than human judgment.
| Element | Detail |
|---|---|
| Questions | 44, across 7 categories |
| Graders | 3 expert audiologists |
| Answer key | Official manufacturer manuals |
| Measures | Clarity, accuracy, repeatability |
Clarity: both AI Hearing Aid Advice models were perfect
Both ChatGPT and Gemini scored a median of 5.00 for clarity, with an interquartile range of 0.00. Essentially every answer, in every category, was rated maximally understandable, and there were no statistically significant differences between the models on this measure anywhere (p > 0.05) [Batuk, ChatGPT and Gemini on hearing aid management, 2026].
On paper, at least, the answers were easy to understand: expert audiologists consistently gave both models the highest clarity ratings. Whether real hearing-aid users find them equally easy to follow was not tested.
Where ChatGPT hearing aid advice diverged from the manual
On medical and technical accuracy, Gemini performed significantly better overall (p < 0.001). The gap was specific rather than global. Gemini pulled ahead in the “Pairing” category (p = 0.03) and in “Before Using a Hearing Aid” (p = 0.03) [Batuk, ChatGPT and Gemini on hearing aid management, 2026].
Which product won is the less interesting half of that result, since version numbers turn over every few months. The interesting half is that two tools nobody could tell apart on clarity were measurably different on accuracy. That creates a potentially important problem: a clear answer can sound authoritative even when its technical accuracy is imperfect.
Repeatability: high, but not perfect
Ask the same question twice and you should get the same answer. Both models came close. Median cosine similarity was 0.91 for ChatGPT and 0.93 for Gemini, with no significant difference between them (p = 0.30). The authors describe this as high semantic consistency, and also as imperfect [Batuk, ChatGPT and Gemini on hearing aid management, 2026]. The answers are stable in substance but not identical in detail.
For a definition of feedback, or a general point about cleaning, that variation is harmless. For a step-by-step sequence where step three matters, a small omission is a different experience.
An evaluation of four chatbots across 100 hearing health questions found a similar pattern. Internal consistency was reasonable for the strongest performer (α = 0.83) but fell below 0.70 specifically for hearing aid, tinnitus, and cochlear implant questions [Pourhoseingholi, Validity, reliability and readability of AI chatbots on hearing loss, 2025]. That study tested ChatGPT-3.5 and Bing AI, so the absolute numbers are dated. The domain-specific pattern is what carries forward.
The case in AI’s favor is stronger than clinicians usually admit
Readers may expect a physician writing this to land on “ask your doctor instead.” The evidence does not support treating that as the whole answer.
Access comes first, and it is closer to arithmetic than to research. A chatbot answers at midnight, on a Sunday, without an appointment, a copay, or a drive. Hearing aids and the professional time around them are expensive, and the questions that generate the most day-to-day frustration are the small ones: pairing, moisture, battery behavior, whistling. Those are exactly the questions people feel least entitled to call a clinic about.
There is early evidence that users find this useful. In an exploratory study, ten adults with hearing loss used an AI hearing health chatbot over two weeks and found it user friendly and helpful for basic support, with hearing aid functionality among the most common topics raised. More experienced users wanted deeper answers [Bennett, Usability and desirability of a hearing health chatbot, 2025]. Ten participants on a purpose-built tool is a preliminary signal, not a settled finding.
AI also adapts on request in a way printed materials cannot. In one evaluation of online health content, only 2% of AI-generated responses met the eighth-grade readability standard by default, no better than the websites they were compared against. When the prompt asked for simpler language, the models dropped their output by nearly six grade levels [Hayes, Readability and quality of website and AI-generated content, 2026]. A separate project fine-tuned a model on validated patient education materials and cut the mean reading grade from 9.8 to 5.8, with a specialist reviewer judging all ten sampled outputs clinically accurate [Mauffrey, Training a bilingual AI model to enhance readability of patient education materials, 2025]. Both come from pediatric orthopaedics, and the second used a fine-tuned model rather than the consumer chatbot most people open.
One wrinkle sits awkwardly across all of this. Pairing is both the most obvious late-night use case and the category where the two models measurably diverged.
Clarity and accuracy are not the same signal
Clarity scored 5.00 across the board. Accuracy did not. The two measures moved independently, and the gap between them is where the concern sits. What follows is interpretation, not a study result.
A well-organized, confident, jargon-free answer reads as verified. Nothing in an AI response marks which sentences came from a manual and which were assembled. A clinician hedges audibly when uncertain, while a chatbot’s tone does not change when it is wrong.
Clinical Perspective
People will ask AI anyway, and the access argument is real, so “stop asking AI” is not advice worth giving. The useful version is narrower. Put the manufacturer and model number in the prompt, because generic hearing aid advice and advice for your device are not the same thing. Ask the model what it is unsure about. Then check anything involving a physical sequence, such as pairing steps, program changes, or cleaning a component, against the manual that came in the box. How confident an answer sounds tells you nothing about whether it is correct. That is a clinical opinion, not a study finding.
The study’s authors reached their own conclusion. Because neither model showed complete stability across all domains, they wrote that these tools cannot currently be recommended as independent digital assistants, and that professional oversight remains necessary to verify accuracy [Batuk, ChatGPT and Gemini on hearing aid management, 2026]. That is their read. The numbers are above, and you are entitled to weigh them yourself.
Key Takeaways
- Three audiologists graded ChatGPT and Gemini on 44 hearing aid questions, using official manufacturer manuals as the answer key.
- Both models scored a perfect median of 5.00 for clarity, with no significant differences in any category.
- Gemini scored significantly higher on medical and technical accuracy overall (p < 0.001), particularly on device pairing.
- Repeatability was high but imperfect for both models (cosine similarity 0.91 and 0.93).
- Hearing aid questions are a category where chatbot reliability has measured lower than other hearing health topics.
- AI answers become substantially more readable when the prompt explicitly asks for simpler language.
FAQ
Can AI answer hearing aid questions accurately? Often, but not uniformly. In the 2026 head-to-head study, accuracy varied by category and by model even though clarity was uniformly high. Questions about general concepts fared better than questions tied to a specific device’s steps.
Is Gemini better than ChatGPT for hearing aid help? In that study, Gemini scored significantly higher on accuracy overall and on pairing specifically. That reflects the versions tested at the time and should not be read as permanent. The useful takeaway is that two models can feel identical while differing in correctness.
Why do I get a different answer when I ask again? Language models do not reproduce answers word for word. Measured similarity was above 0.90 in both models, so the substance usually holds, but details can shift. Asking twice and comparing is a reasonable habit for anything procedural.
How should I word my prompt? Include the manufacturer and model number, state what you already tried, and ask the model to flag anything it is uncertain about. Adding a request for simpler language measurably lowers the reading level of the response.
What should I not rely on AI for? Anything involving a change in your hearing rather than a problem with your device. Sudden hearing loss is the clearest example: clinical practice guidelines call for audiometry as soon as possible, and within 14 days of symptom onset, to confirm the diagnosis [Chandrasekhar, Clinical Practice Guideline: Sudden Hearing Loss (Update), 2019]. New pain, drainage, one-sided symptoms, or persistent ringing belong in the same category. Those are reasons to be seen, not to troubleshoot in a chat window.
References
- Batuk IT, Karakuluk-Celebi I. Evaluating the efficacy of artificial intelligence in audiology: a head-to-head comparison of ChatGPT and Gemini on hearing aid management. Int J Med Inform. 2026;220:106633.
- Pourhoseingholi MA, Killan C, Rafiee S, Hoare DJ, Wray N, Bateman P. Validity, reliability, and readability of Artificial Intelligence chatbots as public sources of information on hearing loss: a comparative evaluation of ChatGPT, Bing, Gemini, and Perplexity. Int J Audiol. 2025;65(6):714-724.
- Bennett RJ, Tsiolkas J, Tagudin J. Usability and desirability of a hearing health chatbot: an explorative study. Int J Audiol. 2025;65(2):233-243.
- Hayes DS, Blank EH, Muchow RD. What Resources Are Available for Parents Online? A Readability, Quality, and Source Evaluation of Website and AI-Generated Content on Inherited Pediatric Orthopaedic Conditions. J Pediatr Soc North Am. 2026;17:100420.
- Mauffrey O, Lashani S, Mitchell SL. Training a Bilingual Artificial Intelligence Model on Pediatric Orthopaedic Resources to Enhance the Readability of Patient Education Materials. J Pediatr Soc North Am. 2025;14:100287.
- Chandrasekhar SS, Tsai Do BS, Schwartz SR, et al. Clinical Practice Guideline: Sudden Hearing Loss (Update). Otolaryngol Head Neck Surg. 2019;161(1_suppl):S1-S45.
Joonpyo Hong, MD is a board-certified otolaryngologist practicing in Korea. This article reflects his clinical interpretation of published research and does not constitute individual medical advice.
For more articles:
https://curiousmd.com/chatgpt-medical-triage-ent-review/
https://curiousmd.com/otc-hearing-aids-2026/
https://curiousmd.com/ai-hearing-aids-2026-ent-review/
https://curiousmd.com/brain-controlled-hearing-aid-2026/
https://curiousmd.com/ai-speech-clarification-hearing-loss/
Link out to:
- Clinical Practice Guideline: Sudden Hearing Loss (Update) — the full AAO-HNSF guideline text, including the 14-day audiometry recommendation cited above (free access).
- AAO-HNS guideline hub and patient handouts — plain-language summaries and printable question lists for patients.
- Over-the-Counter Hearing Aids (NIDCD) — the U.S. National Institutes of Health explainer on OTC devices, including the ear symptoms that warrant seeing a physician.
- Sensorineural Hearing Loss (ENT Health) — AAO-HNSF patient resource on causes, testing, and treatment options.