Operational question
The model is framed as a reviewer triage tool, so the threshold is tied to intervention capacity and policy costs.
Pablo Zavala · AI Safety Evaluation · Research Engineering
DonorsChoose Funding Risk: ML for Targeted Intervention
A model that flags DonorsChoose classroom requests most at risk of going unfunded, so limited reviewer attention can reach under-resourced schools first. The fairness audit reports unequal error rates across school poverty levels as part of the deployment analysis.
Claim precision
- What the evidence demonstrates
- ROC AUC 0.757 on 185,000+ held-out classroom projects
- Capability and evidence frontier
- The model is a policy triage aid; deployment as a funding decision system would require additional validation.
Role: Applied ML analyst: model selection, thresholding, and fairness audit.
How to Inspect This Work
Evidence shown
The evidence card surfaces the held-out sample size, ROC AUC, and fairness concern so readers see both performance and deployment limits.
Deployment limit
The analysis treats unequal error rates across school poverty levels as a policy decision point during model interpretation.
Case Study
Problem
Limited reviewer attention reaches only a fraction of classroom projects, and unfunded projects fall hardest on under-resourced schools.
Setup
The model predicts funding risk from features available when a DonorsChoose request is posted.
Method
Stratified cross-validation selects an XGBoost pipeline; a threshold turns its scores into a bottom-ten-percent review list sized to intervention capacity.
Result
The model reaches ROC AUC 0.757 on more than 185,000 held-out projects.
Verification scope
The fairness audit finds unequal error rates across school poverty levels, so deployment would require policy judgment.
Evidence
The public repository contains the analysis materials and model write-up.
Key Outcomes
- ROC AUC of 0.757 on a held-out test set of more than one hundred eighty-five thousand projects
- Fairness audit found the model misses at-risk projects most often at the highest-poverty schools
- Recommends a bottom-ten-percent review list sized to actual reviewer capacity
Methods
- XGBoost
- Stratified cross-validation
- Threshold selection
- Fairness audit