SUMMARY - Transparency and Explainability
A person is denied a loan and asks why. The bank's representative explains that the decision was made by an algorithm but cannot say what factors drove the denial or how the applicant might improve their chances. A defendant receives a longer sentence based partly on a risk assessment tool that assigns them high likelihood of reoffending, but neither defendant nor judge can examine what variables produced that score.