Training Data Leakage: When Models Remember Too Much | Quiz

Why: When leakage is discovered the incident response plan should be activated including regulatory notification assessment – under GDPR the 72-hour clock starts immediately upon discovery. Context: You need to assess whether the leaked data triggers mandatory breach notification requirements and begin containment. Simply adding filters or retraining takes time you may not have. Remember: Discovery triggers the clock – activate incident response immediately.

Why: GDPR requires notification to regulators within 72 hours once you become aware of a personal data breach – this clock starts immediately when model leakage is discovered. Context: We did not know it memorized that is not a valid defense. Having detection mechanisms and response plans in place is essential for compliance. Remember: The 72-hour clock starts when you discover leakage.

Why: Differential privacy adds calibrated noise during training making it mathematically harder for models to memorize specific examples while preserving overall learning capability. Context: This is considered the gold standard for privacy-preserving machine learning though it comes with accuracy trade-offs. It provides provable privacy guarantees rather than just best-effort protection. Remember: Calibrated noise prevents memorization of specific examples.

Why: Simple anonymization like replacing names with REDACTED leaves structural clues that models can learn from – enabling re-identification attacks that reconstruct identities from patterns. Context: True de-identification requires removing not just identifiers but the patterns that could reconstruct them. Models can infer identity from supposedly anonymous data patterns. Remember: Anonymization hides names but models learn patterns that reveal identity.

Why: The four-stage protection strategy covers data curation before training and training controls during training and output filtering at inference and continuous monitoring post-deployment. Context: No single stage provides complete protection – defense-in-depth across the entire AI lifecycle is required. Organizations that treat it as a set-and-forget problem will eventually face incidents. Remember: Four stages – curation and training and filtering and monitoring.

Why: Short and unique and repeated entries like API keys appearing in multiple code files are far more likely to be memorized than generic paragraphs. Context: When models encounter rare or unique data that appears infrequently or stands out from typical patterns – they encode that data more precisely. Repetition dramatically increases memorization risk. Remember: Rare plus repeated plus unique equals high memorization risk.

Why: Training data leakage occurs when AI models unintentionally memorize and reproduce sensitive information from their training data in their outputs. Context: This includes PII and confidential documents and copyrighted content – creating privacy violations and compliance risks. It is an inherent property of how neural networks learn – not a bug to fix. Remember: Models remember training data and can reveal it in outputs.