Model Drift: Complete Operational Risk Guide | Quiz

Why: Models used in regulated decisions can develop discriminatory biases as population demographics shift – a model fair at deployment may become unfair as underlying data distribution changes. Context: The compliance risk compounds because drift is silent. Decisions accumulate and bias emerges gradually in aggregate outcomes. This can violate fair lending regulations without any intentional discrimination. Remember: Silent drift can create discriminatory bias that violates compliance.

Why: This is concept drift because the relationship between inputs and outputs has changed – the same inputs now mean different things. Context: What used to indicate suspicious is now normal and new attack patterns do not match the old threat signatures. The model is seeing familiar-looking data but the rules of the game have changed. Remember: Same inputs meaning different things equals concept drift.

Why: Canary deployment routes 5-10% of traffic to the new model while monitoring performance – allowing you to roll back immediately if the new model performs worse. Context: This prevents a bad retrain from replacing a working model with something even worse. Never deploy a retrained model directly to 100% of production traffic. Remember: Canary releases protect against bad retrains replacing working models.

Why: When threat detection models drift they stop catching new attack techniques while continuing to run and produce output – providing the illusion of coverage while threats slip through. Context: A network intrusion detection model trained on 2023 patterns may miss 2025 attack techniques entirely. Security teams trust the AI is watching but novel attacks go undetected. Remember: Drifted models provide false confidence – the illusion of security coverage.

Why: The three-layer monitoring strategy provides comprehensive coverage through performance metrics and data distribution monitoring and prediction confidence tracking. Context: Performance metrics are lagging indicators that confirm damage after it occurs. Data distribution monitoring is a leading indicator that provides early warning. Prediction confidence catches both drift types without needing labeled data. Remember: Three layers – performance and distribution and confidence.

Why: Data drift means input distributions have changed (the world looks different) while concept drift means the relationship between inputs and outputs has changed (the world works differently). Context: Both degrade your model but they show up in different monitoring signals and require different detection methods. Data drift can be detected by comparing input distributions while concept drift requires monitoring actual performance. Remember: Data drift – world looks different. Concept drift – world works differently.

Why: Model drift is the gradual degradation of AI model accuracy over time as real-world conditions diverge from the conditions present when the model was trained. Context: This is not a bug or failure but a fundamental property of any AI system that learns from historical data and then faces an evolving future. The world keeps changing while your model stays frozen in time. Remember: Drift is inevitable decay – not a failure but a property of learning from the past.