Pablo Zavala · AI Safety Evaluation · Research Engineering
A predict-then-optimize framework for last-mile delivery, built with Santiago Enríquez as a balanced two-person team for Carnegie Mellon's 94-867 Data to Action course. Gradient-boosted models forecast a central estimate and an 80th-percentile bound for delivery time on a synthetic Kaggle dataset of Amazon-style delivery records; the triage MILP screens orders on the tail forecast, while the simulation MILP prices each order by a classifier's risk of missing its conformally set service promise. In day-by-day simulation, the risk-aware assignment reached 94.8 percent on-time performance at 89.4 percent coverage against a 48.8 percent observed baseline.
In simulation, risk-aware MILP assignment with 240 agents (base risk weight alpha = 1) reached 94.8 percent on-time performance at 89.4 percent coverage, against a 48.8 percent observed baseline on the same synthetic delivery data.
Every headline rate comes from simulation on a synthetic Kaggle delivery dataset; the report flags capacity calibration as the fragile assumption, and the dataset itself stays outside the public repository pending license review, with a README pointer to the download.
Role: Two-person team with Santiago Enríquez under a balanced split recorded in the report: Pablo selected and implemented the predict-then-optimize approach, led the exploratory analysis, and developed the predictive models; Santiago implemented the code, validated the underlying assumptions, and drafted the report both authors finished together.
| Axis | Sample | Evaluator | Result | Limitation |
|---|---|---|---|---|
| Primary output | Scenario runs on a 2,000-order sample plus a day-by-day assignment simulation on synthetic Kaggle delivery data | On-time rate against the 120-minute promise, coverage, and agent utilization per scenario | Risk-aware MILP assignment reached 94.8 percent on-time at 89.4 percent coverage in simulation, versus a 48.8 percent observed baseline. | Simulation on synthetic data; utilization figures depend on capacity calibration. |
| Evidence level | Three committed notebooks with outputs plus the final report PDF | Reader reruns the notebooks after fetching the dataset from Kaggle | Public repository (committed notebooks and report), with the dataset external pending license review. | End-to-end reruns require the external dataset download first. |
| Affiliation | Graduate coursework, October 2025 | Course report title page | Carnegie Mellon, 94-867 Data to Action. | Two-person course project rather than a deployed system. |
| Page updated | This entry and its ledger row | Site claim-source ledger review | July 2026. | Repository contents may grow after this date. |
The notebooks forecast a central estimate beside an 80th percentile of delivery time; from there, the triage MILP screens orders on the tail forecast, and the simulation MILP prices each order by a classifier's risk of missing its conformally set service promise, so intake and assignment decisions respond to uncertainty rather than a single average.
The repository holds the static baseline, dynamic pipeline, and simulation notebooks with their outputs, plus the final report PDF; meanwhile, the Kaggle source dataset stays external pending license review, and the README tells readers where to fetch the file before rerunning.
Read every rate as a simulation output on synthetic data: start with the report's findings pages, then trace the 94.8 percent on-time, 89.4 percent coverage run to the simulation notebook's scenario table and its capacity-calibration caveats.
A delivery network promising a 120-minute window kept that promise on 48.8 percent of historical orders, and averages hide the tail: traffic alone swings delivery times by roughly 46 minutes from light to jammed conditions, while semi-urban zones run slowest at a mean near 239 minutes.
Pablo Zavala and Santiago Enríquez built the framework as a balanced two-person team for Carnegie Mellon's 94-867 Data to Action course, working from a synthetic Kaggle dataset of Amazon-style delivery records and splitting modeling, implementation, and writing evenly.
Engineered spatial, temporal, and agent features feed gradient-boosted models for a central estimate and the 80th percentile of delivery time. From there, the triage MILP screens orders on the tail forecast, while the simulation notebook adds a conformal buffer targeting 90 percent coverage to set each order's service promise, trains a classifier for the probability of keeping that promise, and prices the resulting risk in the assignment MILP; weekly KS tests watch for temporal drift, and a day-by-day simulation replays assignments under agent shift capacities.
On a 2,000-order sample, triage lifted on-time performance to 80.0 percent at 21.3 percent acceptance, static assignment with 300 agents balanced 84.9 percent on-time with 49.8 percent coverage, and the variability-aware simulation with 240 agents reached 94.8 percent on-time at 89.4 percent coverage, versus the 48.8 percent baseline.
Every rate comes from simulation on synthetic data, and the report documents how capacity miscalibration distorts utilization: one calibration pass reported 29 percent utilization at 100 percent coverage, a scaling mismatch rather than true slack.
The public repository commits the three notebooks and the final report PDF; the README credits both authors and points to the Kaggle dataset, which stays external pending license review.