In an era where artificial intelligence (AI) is revolutionizing healthcare, a crucial question arises: Can AI models, trained on sensitive patient data, inadvertently compromise patient privacy? This is the intriguing dilemma that scientists at MIT are exploring.
The Hippocratic Oath, a cornerstone of medical ethics, emphasizes the importance of patient confidentiality. In an age where data is a valuable commodity and cyber threats are rampant, medicine stands as one of the last bastions of privacy. However, a recent paper co-authored by MIT researchers delves into the potential risks associated with AI models trained on de-identified electronic health records (EHRs).
The paper, presented at the 2025 Conference on Neural Information Processing Systems (NeurIPS), highlights a critical issue: AI models, despite being trained on de-identified data, may still memorize patient-specific information. This raises concerns about data leakage and the potential violation of patient privacy. Foundation models, known for their ability to generalize knowledge, can sometimes draw upon a single patient record, potentially exposing sensitive details.
"Knowledge in these models can be a powerful resource, but it can also be exploited by malicious actors," says Sana Tonekaboni, a postdoc at the Eric and Wendy Schmidt Center. She emphasizes the need for practical evaluation steps before releasing such models.
To address this issue, Tonekaboni collaborated with MIT Associate Professor Marzyeh Ghassemi, a leading researcher in robust machine learning for health. Together, they developed a series of tests to assess the risk of data leakage and its potential impact on patient privacy.
"We wanted to ensure that our evaluations were practical and relevant to real-world scenarios," Ghassemi explains. "If an attacker needs a dozen specific details to extract information, the risk is minimal. Our focus is on understanding the practical implications."
The research team's structured tests revealed that the more information an attacker possesses about a patient, the higher the likelihood of data leakage. They also demonstrated methods to distinguish between model generalization and patient-level memorization, crucial for accurate privacy risk assessment.
The paper further emphasizes the varying levels of harm associated with different types of data leaks. Revealing a patient's age or demographics may be less concerning than exposing sensitive medical conditions or diagnoses.
"Patients with unique conditions are particularly vulnerable," Tonekaboni notes. "Even with de-identified data, the type of information leaked can make all the difference."
The researchers plan to expand their work, collaborating with clinicians, privacy experts, and legal professionals to create a more comprehensive framework. "Health data privacy is essential," Tonekaboni asserts. "There's no need for others to know about it."
This research was supported by various institutions and organizations, including the Eric and Wendy Schmidt Center, Wallenberg AI, and the U.S. National Science Foundation (NSF).
As AI continues to shape the future of healthcare, ensuring patient privacy remains a critical challenge. The work of MIT scientists provides valuable insights into this complex issue, offering a step towards safer and more responsible AI integration in medicine.
