Many AI chatbots leak conversation titles and prompts to third-party trackers
The claim
A significant number of chatbot platforms expose user conversation titles and prompts to external trackers
Published as: “Penny for Your Prompts: A Measurement Study of Contextual Leakage in AI Chat Platforms”
So what
The findings show that existing third-party analytics on AI chatbots pose a concrete privacy risk, but platforms can mitigate it by removing or reconfiguring trackers, and users can protect themselves with standard script-blocking tools.
The five-minute version
The study finds that 21 of the 42 AI chatbot platforms leak conversation titles and nine leak verbatim user prompts to third-party trackers, and most of those also expose the user's email, enabling re-identification. Using a combination of Puppeteer network capture and VisibleV8 JavaScript engine instrumentation, the authors attribute each leak to specific user actions across multi-interaction sessions. The evidence is strong, covering both general-purpose and task-specific bots and providing concrete JS-level leakage mechanisms, though it reflects a single US-East viewpoint and a limited time window.
What they found 2
- 21 platforms leak conversation titles and 9 leak verbatim user prompts to third-party trackers
“We find that 21 of these platforms leak conversation titles to third-party trackers, while nine leak verbatim user prompts.”
- Trackers that receive conversational context often also obtain the user's email, enabling linkage to a named individual
“Trackers receiving conversational context frequently also obtain the user’s email, allowing even sensitive disclosures to be tied to a named individual.”
What undercuts it the paper’s own results
- ChatGPT and Grok had third party trackers operating on their platforms that were collecting conversational context. Both platforms have since removed these trackers, which we verified prior to submission.
Stated limits 3
- Experimental Repetitions. Each platform was measured under identical conditions and interaction scenarios, varying only the registration email, between May 6 and May 20, 2026. Our results describe the state of these platforms during that period. Tracker deployments change as platforms add or remove vendors, so the set of platforms leaking conversational context, and the third parties receiving it, may differ outside our measurement window.
- Location and Consent. We measured from a single USEast vantage point with tracking consent accepted.
- Platform Coverage. We study web-based chatbot platforms that offer free account registration, which excludes paid tiers and the growing number of AI chatbots distributed primarily as mobile or desktop applications.
What’s new here 4
- Measures contextual leakage across both general-purpose and domain-specific chatbots, expanding beyond prior work that focused only on general bots.
- Introduces interaction-driven measurement using multi-step sessions rather than a single prompt capture.
- Applies VisibleV8 JavaScript engine instrumentation to attribute leaks to exact user actions and reveal the DOM-level extraction mechanisms.
- Provides responsible disclosure and documents remediation of trackers on platforms like ChatGPT and Grok.
Artifacts & methods 4
- Puppeteer-based network capturesystem
- VisibleV8 (VV8) JavaScript engine instrumentationsystem
- Dataset of 42 AI chatbot platforms (25 general-purpose, 17 task-specific)dataset
- Structured multi-interaction session scenario (account creation, two prompts, navigation, termination)evaluation