Model Drift: Complete Operational Risk Guide | Quiz

Why: This is concept drift because the relationship between inputs and outputs has changed – the same inputs now mean different things. Context: What used to indicate suspicious is now normal and new attack patterns do not match the old threat signatures. The model is seeing familiar-looking data but the rules of the game have changed. Remember: Same inputs meaning different things equals concept drift.

Why: This is data drift (covariate shift) because the distribution of input features has changed – the model is seeing transaction types and payment methods that look fundamentally different from training data. Context: The model is not wrong about what fraud looks like – it is just seeing data that does not match what it was trained to recognize. It is operating out of distribution. Remember: New input patterns from changed world equals data drift.

Why: When detection models drift and generate excessive false positives security analysts become desensitized and start ignoring or auto-closing alerts – causing real threats to get lost in the noise. Context: Alert fatigue is dangerous because analysts stop trusting the system. When a real threat finally triggers an alert it gets the same dismissive treatment as the false positives. Remember: Drift causes false positives which causes fatigue which causes missed breaches.

Why: When threat detection models drift they stop catching new attack techniques while continuing to run and produce output – providing the illusion of coverage while threats slip through. Context: A network intrusion detection model trained on 2023 patterns may miss 2025 attack techniques entirely. Security teams trust the AI is watching but novel attacks go undetected. Remember: Drifted models provide false confidence – the illusion of security coverage.

Why: Accuracy is a lagging indicator that only shows degradation after drift damage has already accumulated for days or weeks. Context: By the time accuracy metrics show problems the damage has already compounded. You need leading indicators like data distribution monitoring to catch drift early before performance degrades. Remember: Accuracy confirms damage already done – you need leading indicators.

Why: Fraud detection and cybersecurity models drift in days or weeks because adversaries actively adapt their tactics to evade detection. Context: Medium-changing domains like customer behavior drift over months while slow-changing domains like scientific modeling drift over years. The speed of drift determines how frequently you need to monitor and retrain. Remember: Adversarial domains drift fastest – attackers adapt continuously.

Why: Data drift means input distributions have changed (the world looks different) while concept drift means the relationship between inputs and outputs has changed (the world works differently). Context: Both degrade your model but they show up in different monitoring signals and require different detection methods. Data drift can be detected by comparing input distributions while concept drift requires monitoring actual performance. Remember: Data drift – world looks different. Concept drift – world works differently.