Healthcare providers' use of AI chatbots: how to prevent data breaches?
More and more healthcare providers are using various forms of artificial intelligence (also known as: artificial intelligence or "AI"). For example, digital assistants such as ChatGPT and Copilot can help with diagnoses, answering questions about treatment, creating referral letters, summarizing large amounts of information or making information more accessible to patients. In this blog, more on the use of AI chatbots by healthcare providers, the consequences of a data breach and how to prevent data breaches when using AI chatbots.
30 October 2024
Especially in healthcare, where the administrative burden is often high, chatbots can save valuable time and provide greater efficiency. At the same time, the use of digital assistants comes with risks that not everyone is aware of. One such risk is a greater likelihood of data breaches occurring. For example, a recent data breach at a general practice occurred after an employee entered patient medical data into an AI chatbot.
Use AI chatbot
Last year, the Autoriteit Persoonsgegevens (AP) received by far most data breach reports of organizations in the health sector, nearly 9,000. In the process, the AP is increasingly receiving reports of data breaches because employees shared personal data with an AI chatbot, such as ChatGPT.
Most parties behind an AI chatbot's software will store all content entered by users for further development and improvement of the chatbot. As a result, the data entered ends up on the servers of those tech companies, often without being clear exactly what those companies will do with that data. The person entering the data into the chatbot is often not sufficiently aware of this. Moreover, in this situation, the person to whom the data relates will not be aware that his/her data is being shared with other users. In fact, there is already a term of its own for such a case: conversational AI leak.
If patient information is shared in this way without consent with a third party who is not directly involved in the performance of the treatment agreement, this causes a breach not only of the General Data Protection Regulation (AVG), but also of medical confidentiality under the Medical Treatment Agreement Act (WGBO), the Youth Act or the Social Support Act 2015. Causing a data breach can also be a disciplinary offence, see for example this statement of the Central Disciplinary Tribunal for Healthcare The Hague. The AP therefore warns against the use of AI chatbots and entering sensitive information such as personal data into them.
Potential consequences and information breach in data breach
A data breach can have major consequences. This is all the more true when patient medical data is involved. This information can end up on the street and be used, for example, for naming and shaming or by cybercriminals for fraud, identity fraud or a phishing attack. Organizations have a duty to report data breaches for a reason. A data breach must be reported to the AP within 72 hours of becoming aware of it. The healthcare provider itself must assess whether in the specific situation it is also obliged to inform the victims of the data leak immediately. This obligation exists when the data breach is likely to pose a high risk to the rights and freedoms of those involved. In case medical data is involved in the data breach, this is almost always the case.
In the practice however, the AP regularly sees - and particularly in healthcare - that affected parties are not notified that their personal data has been leaked because the risks of a data breach are often underestimated. According to some healthcare providers it is not in the best interest of patients to inform them of a data breach. The reasoning behind this is that if certain patients are informed that they are victims of a data breach, their health condition could deteriorate further. Whereas the Medical Treatment Agreement Act contains an exception to the caregiver's duty to inform if doing so would obviously cause serious harm to the patient (also known as the therapeutic exception), no such exception applies when informing about a data breach. Thus, the law does not provide any possibility to deviate from the reporting obligation based on the state of mind of data subjects. As with any other victim, aggrieved patients must also be notified of a data breach if it is plausible that a high risk exists.
Preventing risks in AI chatbots
The use of AI chatbots in healthcare certainly need not be ruled out, but healthcare providers must ensure that employees use such digital assistants responsibly. An AI policy can help in this regard. In an AI policy, a healthcare provider can make its employees aware of the opportunities and risks of using artificial intelligence and also establish rules of conduct for how the use of AI tools is permitted. For example, in a policy that addresses AI chatbots, the healthcare provider could include the following elements:
- Emphasize that employees may use AI chatbots supportively, but employees should not simply rely on the accuracy of AI output and always remain ultimately responsible themselves.
- Emphasize that it is not allowed to enter medical data into an AI chatbot unless it is anonymous, non-patient identifiable information.
- A clear definition of the purposes for which AI chatbots may and may not be used, for example, with respect to diagnosing, summarizing medical articles, in scheduling patient appointments and administrative support.
- An internal reporting procedure for when there is a potential data breach due to the use of an AI chatbot. Such a procedure enables employees to act quickly and appropriately so that serious consequences for affected patients can be avoided as much as possible.
In addition to setting clear rules and guidelines in an AI policy, consideration could be given, for example, to explicitly including AI-related clauses in non-disclosure agreements.
Finally, we mention that the AI Regulation will enter into force in phases beginning in August 2024. The AI Regulation is European legislation on the use of AI systems and applies to both developers and users of AI systems. With respect to chatbots, a transparency obligation applies from August 2026. For a healthcare provider, this transparency obligation means that if, for example, it uses a chatbot that patients can contact themselves - for example, to schedule an appointment - the healthcare provider must then disclose that an AI chatbot is being used and that there is no contact with a real person.
Conclusion
AI chatbots can offer many opportunities in healthcare when it comes to innovation and efficiency, but at the same time they also require a certain degree of adaptability and vigilance from healthcare providers. If healthcare providers are prepared for this in a timely manner and are able to find the right balance between taking advantage of all the possibilities that AI tools such as AI chatbots bring and protecting (medical) data on the other hand, it can definitely be a valuable addition to healthcare.
