AI Transparency & Explainability: Manager’s Guide | QuizBy Eyal Doron / December 6, 2025 / 1 minute of reading AI Transparency & Explainability: Manager’s Guide | Quiz 1 / 7 1. Why is attention visualization for LLMs considered limited as an explanation technique? 1. Attention shows correlation not causation – high attention does not mean that input caused the output 2. Attention mechanisms are not present in modern transformer models 3. Attention visualization only works for small language models 4. Attention patterns cannot be visualized in real-time Correct! WHY: Attention patterns show correlation not causation – high attention on an input does not necessarily mean that input caused the output. CONTEXT: This limitation means attention visualization should be used carefully not as definitive proof of model reasoning. REMEMBER: Attention shows what the model looked at not why it made a decision. 2 / 7 2. According to the EU AI Act what level of transparency is required for high-risk AI systems? 1. Basic disclosure of the model type only 2. No transparency requirements for AI systems 3. Extensive documentation and explanation capability 4. Full source code publication Correct! WHY: The EU AI Act requires extensive documentation and explanation capability for high-risk AI systems. CONTEXT: Requirements scale with risk level and providers must ensure systems can be understood by operators. REMEMBER: EU AI Act equals risk-scaled transparency – higher risk equals more documentation and explanation capability. 3 / 7 3. What makes glass-box models like EBMs and GAMs valuable for high-stakes decisions? 1. They automatically generate GDPR-compliant documentation 2. They require no training data to make predictions 3. They achieve competitive accuracy while remaining fully interpretable by design 4. They are approved by all regulatory bodies for high-risk applications Correct! WHY: Glass-box models achieve competitive accuracy while remaining fully interpretable by design. CONTEXT: Modern interpretable ML has narrowed the accuracy gap making these viable alternatives to black box models for critical applications. REMEMBER: Glass-box models offer both competitive accuracy and built-in explainability. 4 / 7 4. What is the key limitation of post-hoc explanation methods like LIME and SHAP? 1. They require access to training data to function 2. They only work with neural network models 3. They cannot be applied to production systems 4. They show correlations with model behavior not true internal reasoning Correct! WHY: LIME and SHAP are approximations of model behavior not true windows into internal reasoning. CONTEXT: They show correlation with model outputs not necessarily causation or true internal reasoning. REMEMBER: Post-hoc methods are useful tools not ground truth about why the model decided. 5 / 7 5. A credit card company receives complaints about different credit limits for spouses with similar profiles. Without explainability capability what is the primary challenge they face? 1. They cannot comply with PCI-DSS requirements 2. They cannot retrain the model on new data 3. They cannot investigate or defend the algorithm even if it is actually fair 4. They cannot change the credit limit thresholds Correct! WHY: The Apple Card case demonstrated that without explanation capability you cannot investigate or defend your AI decisions even if they are actually fair. CONTEXT: This shows explainability is not just about compliance but about being able to investigate defend and improve AI systems. REMEMBER: No explainability equals no ability to defend your AI even when it is fair. 6 / 7 6. What does SHAP use to determine feature importance in AI predictions? 1. Game theory concepts specifically Shapley values 2. Random sampling of input features 3. Decision tree node splits 4. Neural network attention patterns Correct! WHY: SHAP uses game theory concepts specifically Shapley values to assign importance to each feature for a prediction. CONTEXT: SHAP provides theoretically grounded and consistent feature importance making it a popular choice for explaining black box models. REMEMBER: SHAP equals Shapley values equals game theory for feature importance. 7 / 7 7. Why is the accuracy-explainability trade-off a genuine business challenge for managers? 1. Regulators require all models to have the same level of explainability regardless of accuracy 2. There is no trade-off because modern XAI techniques have solved this problem 3. Explainable models are always more expensive to develop and maintain 4. The most accurate models are often the least explainable creating tension between performance and interpretability Correct! WHY: The most accurate models like deep neural networks are often the least explainable while interpretable models may sacrifice accuracy. CONTEXT: This creates a real business decision about when accuracy is worth the explainability cost. REMEMBER: Match explainability requirements to decision stakes – high consequences need more explainability. Your score isThe average score is 0% Restart quiz Download PDF Please leave this field empty🔐 The AI Security Manager's Newsletter Weekly insights on AI risk management, EU AI Act compliance, and practical security strategies. We don’t spam! Read our privacy policy for more info. Thank you! Please check your inbox to confirm your subscription.
