Model Explainability & Model Risk Management (MRM)

Pngme's latest Model Explainability document can be found here.

This document outlines methodology, performance and stability of Pngme's risk score prediction. The Central Bank of Nigeria does not currently publish or enforce comprehensive regulation on model risk management. This document generally follows guidelines for model risk management as outlined in the US Federal Reserve SR 11-7 Guidelines . Pngme continues to monitor the emerging regulatory space in Nigeria, and expects that any model explainability and risk management that complies with SR 11-7 will also meet emergent regulation.

performance

A lift curve shows a model's ability to separate the best borrowers from the worst borrowers.

The x-axis shows a population of borrowers evenly divided among ten groups (deciles), and ranked from the very best scores (left) to very worse scores (right) based on the model's prediction. The y-axis shows the actual observed loan-overdue rate (bad-rate) for each group. The more powerful the model, the greater the separation between bad-rate for the group on the left and the group on the right.

comparative analysis: pngme vs. a credit bureau

predictive lift

Predictive lift is the ability of a model to separate good borrowers from bad borrowers.

The below plot shows the Pngme lift curve compared to a Nigerian credit bureau's lift curve. It clearly shows that for the same users, Pngme is far more powerful in predicting the difference between goods from bads. Over the entire set of users, the credit bureau can only predict that the worse borrowers are about 10% higher risk than the best borrowers. Pngme, in contrast, can predict a difference of nearly 50% difference in risk between the worse and the best borrowers.

coverage

The below figure shows the proportion of users for which Pngme and/or the credit bureaus have available risk scores. Over half of users (61%) in the population could only have gotten risk scores from Pngme. Pngme can provide risk scores over a much greater population. This is precisely because Pngme has access to data streams that are inaccessible to the credit bureaus, thus has greater visibility into a user's character and financial activities.