AI chatbots are growing in popularity, and their availability is spreading fast. They’re everywhere, on your PC, phone, tablet, and even your IoT smart speaker. Good thing too, since they can be very useful, like for summarizing work docs, helping with math problems, creating complex graphics, and so on.
This wouldn’t be possible without the AI knowing a lot of things…even about the user themselves. But wait a minute, should it actually?
Thing is, many users treat chatbots as their “friend”, a private confidant with useful advice, but they’re anything but that. In fact, you don’t really know where your data might end up, so before you ask for therapeutic, health, legal, or other advice which is personal in nature, think about the implications before you hit “send” on that prompt.
Cybersecurity Awareness Month (every October) is a timely reminder that staying secure involves more than preventing malware; protecting information also plays a major role. In the age of AI chatbots, knowing what not to share has therefore become an important digital habit in its own right.
Key points of this article:
- AI chatbots can remember and reuse more of your conversations than you may realize.
- Some details should never enter a prompt, no matter how useful the response might be.
- A quick pause before prompting can prevent sensitive information from going where it should not.
- Anonymization can make prompts safer, but context may still reveal more than intended.
- Safer AI use starts with approved tools, careful prompts, and the assumption that nothing shared is truly private.
AI chatbots remember more than you think
Depending on your platform of choice, AI chatbots like ChatGPT, Claude, or Copilot usually store inputs so that they may be:
- searched for in the future by the user
- be kept as part of broader context about an individual (memory) or their work
- reviewed by the platform owner for Terms of Service (ToS) violations and to improve safety
- used to improve services (future development), and more
Most of it is about making AI use more convenient. However, at the core of it is making models better—whether through training, testing, or fine-tuning, with user prompts supplying diverse human input. That, and usage statistics can help providers understand where more specific models or capabilities are needed. Still, this doesn’t mean every prompt is automatically used for training, as practices vary between platforms and account settings.
Also, information may also be exposed through provider-side security incidents, shared chat links, or integrations that pass prompts to other services.
A bit of a disclaimer though: not every AI platform is the same. Some actually offer zero data retention (ZDR) and promise not to train on user data. Still, before committing to any particular model, read the company’s data handling practices and check the privacy settings to know what to expect.
Five things you should never share
Even though privacy and data sovereignty-conscious platform options exist, users should still exercise caution. Policies, terms of use, and company priorities change, so one day your data could as well start being fed into the AI monster for training. Here’s some advice on what’s better being kept private:
- Passwords and credentials: Usernames, account passwords, MFA recovery codes, security questions for account recovery/access
Chatbot memory might seem convenient, but storing or reviewing your credentials could easily expose them during random prompt review processes, for example.
- Sensitive personally identifiable information (SPII): ID and passport details, home addresses, financial details.
Technically some of this might be traded for to an AI provider when paying for a subscription, but this information is usually heavily protected due to regulatory compliance reasons. The same is not true when ingested by a chatbot though, and when accessed by someone, these could aid identity theft.
- Confidential work information: Internal reports, customer data, product plans, source code, non-public business information
It might seem productive to have a confidential doc summarized by AI. However, unless the tool is approved for that type of information and covered by your employer’s agreement with the provider, uploading it could violate company policy, contractual duties, or data-protection requirements—and may need to be treated as a security incident.
As an example, in 2023, Samsung employees reportedly pasted proprietary source code, semiconductor testing information, and notes from an internal meeting into ChatGPT while using it to help with everyday work tasks. The incident showed how easily a productivity shortcut can move confidential information beyond an organization’s control.
- Medical and highly sensitive personal data: Health records, test results, treatment details and more.
Sure, people share sensitive information when seeking advice on difficult life situations. But AI is not a healthcare provider, and it won’t be able to treat your issue like an actual doctor would, bound by ethics and the law to keep your data safe.
- Information about other people: Colleague’s details, customer records, private conversations, contact information, and more.
Protecting privacy means safeguarding other people’s data, too. For one, any exposure is dangerous, and by doing so without one’s knowledge, you’re not only breaking a few ethical and moral rules, but also potentially putting these people at risk of something very bad down the line if said data should appear somewhere unintended.
For instance, in 2025, thousands of ChatGPT conversations shared through public links appeared in search results, with some revealing personal or professional details. The chats had not been leaked from private accounts, but the incident showed how a seemingly simple sharing option could make a conversation far more public than users expected.
Moreover, remember, a prompt is not just the text you type in. Uploaded documents, screenshots, images, voice recordings, and information pulled in through connected apps may contain the same sensitive details—sometimes hidden in metadata, comments, or background content…all of which the smart AI can see and use, so don’t be surprised when your AI pulls up a resolved Word comment for context.
Think before you prompt
Before you share anything, always ask yourself:
- Is this public information?
- Would I share this in a public forum?
- Am I authorized to share it?
- Would I be comfortable if it appeared in a data breach?
If the answer is no, then don’t paste it into a chatbot, ever.
Private chats are not necessarily private
Private, incognito, or temporary chat modes can reduce exposure by keeping a conversation out of your visible history or excluding it from model training. But the word private can be misleading: the provider may still retain the conversation for a limited period to detect abuse, investigate security issues, or meet legal obligations. These modes also do not protect information sent through plugins, connected apps, shared links, or other third-party services.
Use AI safely
If you really want to share something with your chatbot though, there’s a way around to make your prompt “safer” by:
- Removing names and identifiers
- Using fictional or anonymized examples instead of real issues
- Summarizing sensitive docs yourself before asking AI for help
- Using approved enterprise AI tools with data protection when available at work
Note, approval is not a blank check. Check which data classifications the tool is permitted to process, whether chat history or memory is enabled, and whether external plugins, connectors, or web-search features can send information beyond the organization’s-controlled environment to stay extra safe.
Mind you, a smart chatbot could very well infer individually relevant details based on your prompting history even if you anonymize data. It’s just the nature of machine learning, and while you might think that deleting the conversation deletes its trace, that may not necessarily be the case. Deleting the conversation may reduce exposure, but it should not be treated as proof that every copy is gone (think back to data retention and how AI is trained).
So, If you realize you have shared something sensitive, act based on what was exposed. Replace credentials immediately, delete the chat and clear any saved memory where possible, and report work, customer, or third-party data through the appropriate security or privacy channel.
Users can reinforce these habits with security controls designed specifically for AI use. ESET’s AI Security capabilities analyze prompts and responses in real time, helping prevent sensitive information from being uploaded to public chatbots, while also detecting malicious links, scripts, and other risky content.
Conclusion: Treat AI like a public forum
Sure, AI can boost productivity and creativity, but users should apply the same caution they would use when sharing information with an acquaintance: good for surface-level conversations, but if you don’t want to embarrass yourself, keep most details private.
FAQ: AI chatbot use
Are AI chatbot conversations private?
Not necessarily. How conversations are stored, reviewed, or used varies by provider, product, account type, and privacy settings. Treat a consumer chatbot as a third-party service rather than a private confidant and check its current data-handling terms before sharing anything sensitive.
Does deleting a chat erase the data completely?
Deleting a conversation usually removes it from your visible history, but it may not immediately erase every retained copy or operational log. Retention periods and exceptions differ between services, so consult the provider’s privacy and data-retention information.
Is anonymizing information enough to make a prompt safe?
It reduces risk, but it is not foolproof. A combination of details, context, or earlier prompts may still reveal the person or organization involved. Remove unnecessary specifics, use fictional examples where possible, and do not submit information you are not authorized to share.
Are enterprise AI tools safer for work information?
Approved enterprise tools may provide stronger contractual protections, administrative controls, and restrictions on using customer data for model training. However, employees should still follow their organization’s data confidentiality/use policies and only share information the tool is approved to process.
What should I do if I have shared sensitive information?
Delete the conversation where possible, clear any saved memory, and revoke connected-app access if relevant. Also, replace any exposed passwords, recovery codes, or other credentials immediately. If company, customer, or third-party data was involved, report the incident to your employer or security team rather than trying to handle it alone.
