Machine Unlearning on AWS: Safeguarding Sensitive Healthcare Data (Part 1)
This post was co-written between Gustavo Romero, Encora and Prateek Agrawal, AWS. As healthcare organizations increasingly adopt large language models (LLMs) to enhance clinical operations and patient engagement, a new challenge has emerged: what happens when an AI model learns something it shouldn’t?
From protected health information (PHI) under HIPAA to personally identifiable data under GDPR, LLMs can sometimes memorize sensitive information. Unlike traditional software, these models aren’t designed to "forget" on demand. Removing data from a trained model is like trying to extract an ingredient from a baked cake—extremely difficult and often impractical.
That’s where machine unlearning comes in. It’s an emerging field focused on removing specific knowledge from AI models without requiring full retraining. In this blog series, we’ll explore how AWS services and custom workflows can help organizations implement machine unlearning at scale—starting with targeted fine-tuning using anti-data.
Why does this matter? In healthcare, under HIPAA, any inadvertent memorization of PHI in an LLM could lead to unauthorized disclosures. Under GDPR’s “right to be forgotten,” individuals can demand that their personal data be removed from AI models.
Out-of-the-box, LLMs lack a "delete" function, and retraining large models is costly and time-consuming. While major LLM providers don’t yet offer native machine unlearning, AWS services can be used to build scalable, compliant solutions.
We’ll cover three key architectural approaches of unlearning to meet regulatory requirements.
The Compliance Challenge: Sensitive Data in LLMs
LLMs are trained on vast text corpora and can inadvertently memorize verbatim data – including names, addresses, medical record numbers, or entire documents. In healthcare use cases, this is especially dangerous. For example, imagine an LLM-based clinical assistant that was fine-tuned on real patient notes. If a user uses prompts injections, the model might regurgitate parts of a patient’s record. This would be a severe HIPAA violation. Likewise, an LLM might have ingested copyrighted medical texts or proprietary drug databases – content that must be unlearned if usage rights are revoked or to comply with GDPR erasure requests.
Fig 1.0 - PII exposure via fine tuning
Compounding the challenge, removing knowledge from an LLM is much harder than adding knowledge. Researchers aptly compared it to removing ingredients from a cake, as opposed to adding new flavors via icing. A naïve solution is to retrain the model from scratch on a filtered dataset (omitting the sensitive data). But for large models, retraining can be astronomically costly and time-consuming. We need more efficient strategies to overcome these challenges.
In the next sections, we dive into three such approaches and how to implement them on AWS, balancing unlearning precision, cost, and compliance.
Key Approaches to Machine Unlearning on AWS
Modern machine unlearning techniques for LLMs generally fall into two categories
Model-Based Unlearning:
Directly modifying the model’s parameters to forget specific information (e.g. through fine-tuning or editing weights).
Data-Based Unlearning:
Altering how data is fed to the model (e.g. using retrieval augmentation or prompt filters so the model never sees or uses the sensitive data).
Our focus is on model-based approaches, since they actually expunge information from the model’s memory – which is crucial for compliance (you can’t accidentally leak what the model truly doesn’t know). In this blog series, we’ll cover three architecture patterns:
Fine-Tuning with “Anti-Data”
– A targeted fine-tune to make the model forget or suppress certain knowledge.
Surgical Model Editing (e.g. ROME/MEMIT)
– Directly locating and modifying the model’s internal weights associated with a specific fact or set of facts.
Full Retraining (Selective)
– Re-training or fine-tuning the model from scratch with the sensitive data removed, using LLMOps pipelines on AWS to manage versioning and lineage.
Each approach has trade-offs in precision vs. cost, speed vs. thoroughness, and will appeal to different use cases. In practice, organizations may combine these methods (for example, using quick edits as an immediate fix and scheduling a retrain for a permanent solution). In this post, we’ll focus on the first approach, which is fast, cost-effective, and highly adaptable.
Approach 1: Fine-Tuning with “Anti-Data” (Targeted Unlearning Fine-Tune)
One intuitive way to make an LLM forget something is to fine-tune it with new data that “un-teaches” the targeted knowledge. This “anti-data” might consist of perturbations or replacements of the original sensitive information, or instructions that explicitly tell the model to avoid certain content. Fine-tuning with anti-data is an effective method to reduce or eliminate a model’s recall of specific facts. The idea is simple: you “un-teach” the model by fine-tuning it with data that either:
Omit
the sensitive information
Contradict
it with misleading or neutral alternativesInstruct the model to avoid mentioning it
In practice, researchers have had success with this approach and demonstrated “approximate unlearning” by fine-tuning Llama-2 (7B) to forget the content of the Harry Potter books. Impressively, this was achieved in about 1 GPU-hour of fine-tuning (versus 184k GPU-hours to originally train the model), and the model’s performance on other tasks remained largely unaffected.
Below is a high-level pipeline to implement such targeted unlearning fine-tuning on AWS.
Fig 2.0 - Fine-tuning pipeline for targeted unlearning on Amazon SageMaker
Fine-tuning pipeline for targeted unlearning. In this pipeline, we use Amazon SageMaker to run a fine-tuning job on the original model with a curated “forgetting” dataset. After fine-tuning, automated evaluation steps test the model to ensure the target data is truly gone (and that general performance is retained). If the model passes the tests (e.g., it no longer outputs the sensitive content in question), the new model version is registered and deployed to production. This continuous integration/continuous delivery pipeline helps ensure compliance before a model update is fully released.
How it works:
You can set up this workflow using Amazon SageMaker and related services:
Step 1: Identify What to Forget
Locate prompts, training examples, or behaviors the model needs to unlearn (e.g., a specific patient’s name).
Step 2: Build the Anti-Dataset
Create training samples where the sensitive data is missing, incorrect, or neutralized.
We would create an anti-dataset that might include prompts about that patient with no address in the responses, or the address replaced with generic text. Another technique is to generate misleading or confounding examples (the model’s own hallucinations can be leveraged for this) to reduce the model’s confidence in the original memory.
Step 3: Launch a Amazon SageMaker Fine-Tuning Job
Use Amazon SageMaker to fine-tune the model with anti-data. The original model weights are loaded, and train on the anti-data for a short duration for efficient updates, consider using LoRA (Low-Rank Adaptation) to update only a subset of model parameters.
Step 4: Test for Successful Unlearning
Prompt the model with related questions to confirm it no longer recalls the data or, requests are refused.
Running standard benchmark tests to ensure we haven’t broken unrelated knowledge (avoiding “catastrophic forgetting” where the model forgets too much).
Step 5: Register and Deploy the New Model
Use Amazon SageMaker Model Registry to manage model versions. Archive previous versions for audit purposes.
Pros of fine-tuning for unlearning:
Speed and Cost:
It’s much faster and cheaper than full retraining.
Granularity:
You can target very specific data or behavior to forget. The rest of the model’s knowledge is largely preserved if done carefully.
Feasible with AWS Managed Services:
Amazon SageMaker enables to spin up the necessary GPU instances for a short fine-tune job.It can further integrate with Amazon SageMaker Pipelines, Amazon CloudWatch for a controlled MLOps process while AWS IAM security features.
Compliance Response Time:
This method enables responding to a GDPR deletion request or HIPAA incident– possibly within hours or days – by issuing a model update, rather than saying “we’ll retrain in a few months.” timely removal, limits the window of exposure.
Cons/challenges:
Incomplete Unlearning:
Naively fine-tuning on a small anti-dataset might not completely scrub the information. Research shows that fine-tuned “unlearned” models can still retain latent traces of the data. In one study, even after unlearning, models still inadvertently retained around 21% of the original knowledge. Rigorous testing is needed to ensure compliance.
Catastrophic Forgetting:
If the unlearning data or method is too broad, the model may forget more than intended – including
dropping accuracy on unrelated tasks. We want to forget only the target data and nothing else. This often requires careful crafting of the fine-tuning set and possibly regularization techniques to preserve other knowledge.
Iterative Process:
It may take several iterations of fine-tuning and testing to get it right. Each iteration should be tracked (which data was used, what the results were) for audit purposes.
No Guarantee of Irreversibility:
Fine-tuning can reduce the model’s tendency to output something, but it doesn’t
verifiably
erase the internal representation.
Use case relevance:
For many HIPAA/GDPR scenarios, fine-tuning is a practical first step. Example: A patient invokes GDPR to remove their data. The ML team can locate all prompts/answers in the training set about that patient, create an anti-data fine-tuning set (e.g. ask the model about the patient and train it to respond “I don’t know”), and push an updated model timely.
Conclusion: Looking Ahead to Forgetful and Compliant AI
Machine unlearning is no longer just academic—it’s a practical requirement in regulated sectors like healthcare. Using AWS tools such as AmazonSageMaker, teams can implement agile, cost-effective workflows to selectively remove knowledge from LLMs.
To recap, fine-tuning with anti-data offers a quick, targeted way to scrub specific knowledge, using AWS’s scalable training infrastructure to apply “negative” examples that reduce the model’s recall of sensitive info. It’s cost-effective and fast, but must be carefully validated to ensure complete removal.
For healthcare leaders and AI Service providers, investing in machine unlearning capabilities now is a strategic move. It mitigates legal risks (avoiding hefty fines or breaches), and it builds trust with patients and regulators by showing that your AI is under control. “Machine unlearning technology will become a must as AI regulation evolves” By following the patterns described here one can stay ahead of the curve that large language models remain compliant, up-to-date, and worthy of the immense trust we place in them in healthcare.
Ready to build forgetful AI on AWS?
Review your compliance obligations (HIPAA, GDPR)
Identify LLM use cases where data erasure may be needed
Start testing targeted fine-tuning with Amazon SageMaker
Stay tuned for Part 2, where we dive into surgical editing techniques like ROME and MEMIT.
About the authors
Gustavo Alejandro Romero Sanchez is a Principal Cloud Solutions Architect at Encora. He delivers guidance on Cloud Security, Generative AI solutions, DevSecOps, and scalable architecture to Encora’s customers.
Prateek Agrawal is a Sr. Partner Solutions Architect with Amazon Web Services. He provides architecture guidance on enterprise cloud adoption, migration, and AI strategy to AWS partners and customers.