Training Data Poisoning: Complete Defense Framework -Quiz

Why: This is most likely a sleeper attack because the backdoor remained dormant through testing and months of production, then activated based on a specific condition (the merchant code trigger). Context: Sleeper attacks are designed to pass all testing and only activate when the attacker-chosen condition is met – often a date or specific input pattern. Remember: Dormant for months then suddenly activates – classic sleeper attack pattern.

Why: The article emphasizes that no single defense is sufficient because sophisticated attacks are designed to evade individual controls – defense requires multiple layers working together. Context: Effective defense needs provenance tracking AND anomaly detection AND robust training AND continuous validation – not any one technique alone. Remember: No single defense works – layered security is required.

Why: The Pravda network created approximately 3.6 million articles specifically designed to be scraped by AI training pipelines to inject specific narratives and biases into language models. Context: This attack exploited the fundamental vulnerability that most large language models are trained on web-scraped data with no reliable way to verify authenticity at scale. Remember: Pravda – 3.6 million fake articles to poison the entire web training pipeline.

Why: Fine-tuning uses smaller datasets with higher per-sample influence on model behavior, so a handful of poisoned samples can override base model behavior entirely. Context: The smaller the dataset, the more impact each individual sample has – making fine-tuning particularly vulnerable to targeted attacks. Remember: Smaller dataset means bigger impact per sample – fine-tuning is high leverage for attackers.

Why: Unlike traditional security threats that can be patched, training data poisoning requires retraining the entire model to fix because the corruption is baked into the model weights. Context: You cannot simply remove malicious data after training – the model has already learned from it and encoded those patterns permanently. Remember: You cannot patch poisoning – only retrain from scratch.

Why: Integrity attacks create hidden backdoors that let attackers control the model while it appears normal in all other cases – the model passes all standard accuracy tests. Context: The attack is invisible until the attacker activates the specific trigger, which is why normal testing and monitoring will not detect it. Remember: Backdoors pass all tests – the model works perfectly until the attacker wants it not to.

Why: Research shows poisoning attacks can achieve over 92% success rates with less than 0.1% of training data contaminated. Context: This demonstrates how little poisoned data is needed to compromise an entire model – even a tiny fraction affects every future interaction. Remember: Less than 0.1% contamination can achieve 92% success – small poison, big damage.