A Warning: Your Health Data is at Risk!
In a world where AI is rapidly advancing, we find ourselves at a crossroads. Over 230 million people are turning to chatbots like ChatGPT for health advice, but is this a wise decision? OpenAI, the company behind ChatGPT, paints a picture of this chatbot as a trusted 'ally', offering assistance with insurance, paperwork, and self-advocacy. But here's where it gets controversial: in exchange for these services, they want access to your most sensitive medical information.
Health and wellness have become a battleground for AI companies, testing the limits of user acceptance. This month, two industry giants, OpenAI and Anthropic, made bold moves into the medical domain. OpenAI introduced ChatGPT Health, a dedicated space within ChatGPT, promising a secure and personalized environment for health-related queries. Anthropic followed suit with Claude for Healthcare, a product they claim is 'HIPAA-ready' for use by hospitals, health providers, and consumers.
OpenAI actively promotes the sharing of sensitive health data, including medical records and personal wellness information, with ChatGPT Health. They assure users of confidentiality and promise not to use this data for AI model training. However, the company's own actions raise concerns. They launched a similar-sounding product, ChatGPT for Healthcare, with tighter security protocols, almost simultaneously. This product is designed for businesses and clinicians, and it's easy to confuse the two, assuming the consumer version offers the same level of protection.
Even if we trust a company's promise to safeguard our data, there's no guarantee. As Carmel Shachar, an assistant clinical professor of law, points out, 'There’s very limited protection. Some of it is their word, but they could always go back and change their privacy practices.'
The lack of comprehensive privacy laws leaves users vulnerable. As Sara Gerke, a law professor, explains, data protection for AI tools like ChatGPT Health 'largely depends on what companies promise in their privacy policies and terms of use.'
But it's not just about privacy. Medicine is a heavily regulated field for a reason. Errors can be deadly, and chatbots have a history of providing false or misleading health information. For instance, a man developed a rare condition after ChatGPT suggested replacing salt with sodium bromide, a historical sedative. Google's AI Overviews also gave dangerous advice to pancreatic cancer patients, recommending they avoid high-fat foods, the opposite of what's recommended.
OpenAI claims that ChatGPT Health is not intended for diagnosis or treatment, but their actions suggest otherwise. They've highlighted health as a major use case and even invited a cancer patient to discuss how the tool helped her understand her diagnosis. This raises questions about the regulatory challenges posed by chatbots and whether they should be classified as medical devices.
Despite disclaimers, OpenAI's efforts to present ChatGPT as a capable medic may undermine user trust. As Hannah van Kolfschooten, a researcher in digital health law, says, 'When a system feels personalized and has this aura of authority, medical disclaimers will not necessarily challenge people’s trust in the system.'
AI companies are hoping to gain our trust as they enter the healthcare market. While this could address health inequalities and improve access to care, we must ask: has this industry, known for its 'move fast and break things' mentality, earned our trust with our most sensitive information?
This is a crucial conversation, and we encourage you to share your thoughts and concerns in the comments. Do you think AI chatbots can be trusted with our health data? Is the potential benefit worth the risk?
