Operational question
The model is framed as a reviewer triage tool, so the threshold is tied to intervention capacity rather than a generic accuracy target.
Pablo Zavala · AI Safety Evaluation · Research Engineering
DonorsChoose Funding Risk: ML for Targeted Intervention
A model that flags DonorsChoose classroom requests most at risk of going unfunded, so limited reviewer attention can reach under-resourced schools first. The fairness audit reports unequal error rates across school poverty levels rather than presenting only an average score.
ROC AUC 0.757 on 185,000+ held-out classroom projects
The model is a policy triage aid, not a deployment-ready funding decision system.
Role: Applied ML analyst: model selection, thresholding, and fairness audit.
How to Inspect This Work
Evidence shown
The evidence card surfaces the held-out sample size, ROC AUC, and fairness concern so readers see both performance and deployment limits.
Deployment limit
Unequal error rates across school poverty levels are treated as a policy decision point, not as a footnote after model selection.
Case Study
Problem
Limited reviewer attention cannot reach every classroom project, and unfunded projects fall hardest on under-resourced schools.
Setup
The model predicts funding risk from features available when a DonorsChoose request is posted.
Method
An XGBoost pipeline is selected under stratified cross-validation, then thresholded into a bottom-ten-percent review list sized to intervention capacity.
Result
The model reaches ROC AUC 0.757 on more than 185,000 held-out projects.
Limitation
The fairness audit finds unequal error rates across school poverty levels, so the model is not presented as deployment-ready without policy judgment.
Evidence
The public repository contains the analysis materials and model write-up.
Key Outcomes
- ROC AUC of 0.757 on a held-out test set of more than one hundred eighty-five thousand projects
- Fairness audit found the model misses at-risk projects most often at the highest-poverty schools
- Recommends a bottom-ten-percent review list sized to actual reviewer capacity
Methods
- XGBoost
- Stratified cross-validation
- Threshold selection
- Fairness audit