Pablo Zavala · AI Safety Evaluation · Research Engineering

What Moves the Price of an LA Home?

A University of Chicago Booth big-data study, co-authored with Will Sigal, that prices 3,804 Los Angeles County single-family homes from structural and neighborhood features, then audits its own headline. A tuned stacked ensemble reaches out-of-sample R-squared 0.621 under random cross-validation and falls to 0.431 when scoring moves to held-out geographic blocks; on the separate leave-one-region-out spectrum, tree-ensemble skill decays toward zero while the humble regularized linear model holds up best across space. Cross-fitted Double/Debiased ML puts the direct coastal premium near +29.8% per square foot, while the naive private-school premium collapses to a statistical zero.

Across roughly ten tuned learners, random-CV R-squared 0.621 falls to 0.431 under spatial-block validation, and a leave-one-region-out spectrum sends tree-ensemble skill toward zero; Double ML estimates a +29.8% direct coastal premium (95% CI [0.16, 0.33] log points) and shrinks the naive private-school gap to +0.7%.

Public reproducible repo + working paper

Causal readings hold only under ignorability, the coastal estimate rests on 100 treated homes, and third-party terms keep home-level rows out of the repository, so a full rerun starts from the cleaned table the committed ingestion code rebuilds.

Role: Joint coursework with Will Sigal: shared design, modeling, and writing across the analysis package and the 22-page working paper.

Evaluation Card

Primary output

Sample

3,804 single-family listings across 152 LA-County cities, one cleaned modeling table

Evaluator

Pooled out-of-fold R-squared under random 5-fold, spatial-block, and leave-one-region-out cross-validation; nested-CV tuned sweep with stacking

Result

A tuned stacked ensemble leads random CV at R-squared 0.621; spatial blocks cut its score to 0.431 while the tuned elastic net posts the best spatial figure at 0.448; on the leave-one-region-out spectrum, the regularized linear model extrapolates best.

Limitation

One county and one listing snapshot; spatial blocks approximate, rather than reproduce, true out-of-region deployment.

Causal audit

Sample

Four treatments: coast within half a mile (100 treated), private school within 2 miles, any foreclosure within 1 mile, distance per mile

Evaluator

Cross-fitted Double/Debiased ML with gradient-boosted nuisances; spatial block bootstrap for 95% CIs; control-set sensitivity

Result

Coast +29.8% (CI [0.16, 0.33] log points); private school +0.7% (CI [-0.11, 0.11] log points); foreclosure -7.0% (CI [-0.14, -0.01] log points), read as suggestive; with structural controls only, the total coastal premium reads +79.4%.

Limitation

Confounding-adjusted observational estimates, causal only under ignorability given the controls.

Evidence level

Sample

Public repository with the working paper, figures, tables, tests, and CI

Evaluator

make all regenerates every figure and table offline; macros.tex, results.json, and results_full.json lock the manuscript to pipeline output

Result

Every number quoted on this page appears in the committed results files, the generated macros, the paper tables, the README, or the hosted paper PDF.

Limitation

Home-level rows stay withheld under third-party terms; rebuilding the cleaned table needs the raw sources plus the committed ingestion code.

Affiliation

Sample

University of Chicago Booth, BUSN 41201 (Big Data), May 2024

Evaluator

Paper title page and repository README

Result

Graduate coursework co-authored by Pablo Zavala and Will Sigal, following the Taddy Business Data Science toolkit.

Limitation

Course handouts, prompts, and grading context stay private.

Page updated

Sample

July 2026

Evaluator

Site content ledger

Result

Entry drafted July 10, 2026 from the public repository, its committed results, and the working paper.

Limitation

Numbers reflect the committed pipeline outputs rather than a fresh rerun.

Evaluation axes with sample size, evaluator, result, and limitation.

Axis	Sample	Evaluator	Result	Limitation
Primary output	3,804 single-family listings across 152 LA-County cities, one cleaned modeling table	Pooled out-of-fold R-squared under random 5-fold, spatial-block, and leave-one-region-out cross-validation; nested-CV tuned sweep with stacking	A tuned stacked ensemble leads random CV at R-squared 0.621; spatial blocks cut its score to 0.431 while the tuned elastic net posts the best spatial figure at 0.448; on the leave-one-region-out spectrum, the regularized linear model extrapolates best.	One county and one listing snapshot; spatial blocks approximate, rather than reproduce, true out-of-region deployment.
Causal audit	Four treatments: coast within half a mile (100 treated), private school within 2 miles, any foreclosure within 1 mile, distance per mile	Cross-fitted Double/Debiased ML with gradient-boosted nuisances; spatial block bootstrap for 95% CIs; control-set sensitivity	Coast +29.8% (CI [0.16, 0.33] log points); private school +0.7% (CI [-0.11, 0.11] log points); foreclosure -7.0% (CI [-0.14, -0.01] log points), read as suggestive; with structural controls only, the total coastal premium reads +79.4%.	Confounding-adjusted observational estimates, causal only under ignorability given the controls.
Evidence level	Public repository with the working paper, figures, tables, tests, and CI	make all regenerates every figure and table offline; macros.tex, results.json, and results_full.json lock the manuscript to pipeline output	Every number quoted on this page appears in the committed results files, the generated macros, the paper tables, the README, or the hosted paper PDF.	Home-level rows stay withheld under third-party terms; rebuilding the cleaned table needs the raw sources plus the committed ingestion code.
Affiliation	University of Chicago Booth, BUSN 41201 (Big Data), May 2024	Paper title page and repository README	Graduate coursework co-authored by Pablo Zavala and Will Sigal, following the Taddy Business Data Science toolkit.	Course handouts, prompts, and grading context stay private.
Page updated	July 2026	Site content ledger	Entry drafted July 10, 2026 from the public repository, its committed results, and the working paper.	Numbers reflect the committed pipeline outputs rather than a fresh rerun.

How to Inspect This Work

One score, three answers

The paper scores its learners under regimes that answer different questions. Random cross-validation (R-squared 0.621 for the tuned ensemble) prices a home whose neighbors appear in training; spatial-block validation (0.431) prices a partly held-out region; and a leave-one-region-out spectrum over LASSO, random forest, and XGBoost prices genuinely unseen parts of the county, where tree-ensemble skill decays toward zero. The spread between regimes measures how much of the headline rests on interpolation.

Causation checked against association

Every causal claim sits beside its naive counterpart. The raw coastal gap of 0.904 log points shrinks to a Double ML estimate of 0.261; the raw private-school gap of 0.442 shrinks to 0.007 with an interval straddling zero; and the foreclosure association flips sign, from a positive raw gap to -7.0% after adjustment, an estimate the paper reads as suggestive because the interval ends near zero.

Reader check

With the cleaned modeling table in place (home-level rows stay withheld under third-party terms), make analysis rebuilds the baseline figures and tables offline in about two minutes, make all regenerates everything, and the committed results record a 21.9-minute full tuned-plus-causal run. Generated macros lock every number in the manuscript to pipeline output, tests guard the data contracts while CI runs the pure-logic suite, and the data card records provenance for each withheld source, so each quoted figure traces to committed code.

Case Study

Problem

Price per square foot varies several-fold across Los Angeles County, often within a few miles, and a single cross-validated score flatters any learner on spatial data; the study therefore needs an evaluation protocol that separates transferable structure from interpolation between near neighbors.

Setup

The analysis table holds 3,804 single-family listings across 152 cities, each enriched through spatial joins with seven neighborhood layers: school counts, tract-level violent crime, census income and density, park acreage, top-rated restaurants, 2021 foreclosure filings, and coastal distance. The target, log price per square foot, tames a heavy right tail and lets coefficients read as approximate percentage effects.

Method

A tuned sweep of roughly ten learners, capped by a stacked ensemble, scores on pooled out-of-fold predictions under random 5-fold and spatial-block regimes, while a separate leave-one-region-out spectrum stresses the untuned LASSO, random forest, and XGBoost trio. A hedonic OLS with HC3-robust standard errors, a LASSO sparsity check, and a held-out permutation-importance pass on the random forest handle association; cross-fitted Double/Debiased ML with a spatial block bootstrap handles causal adjustment; Moran's I audits residual clustering.

Result

The stacked ensemble reaches R-squared 0.621 under random CV and 0.431 under spatial blocks; on the leave-one-region-out spectrum, XGBoost falls to 0.007 at half the county held out while LASSO holds 0.276. Double ML puts the direct coastal premium at +29.8% per square foot (95% CI [0.16, 0.33] log points) against a naive gap of 0.904, shrinks the private-school premium to a statistical zero, and flips the foreclosure externality to -7.0%, a suggestive reading with an interval ending near zero.

Limitation

Causal estimates stay observational, valid under ignorability alone, and the coastal contrast rests on 100 treated homes; out-of-sample residuals keep spatial structure (Moran's I 0.056 under random CV, 0.346 under spatial CV), which points toward explicit spatial-error modeling as the next step.

Evidence

With the cleaned table in place, the public repository regenerates every figure and table offline through make all, locks the manuscript's numbers through generated macros, and ships tests plus a pure-logic CI suite; the data card records provenance and licensing for each withheld source.

Key Outcomes

• Random-CV R-squared 0.621 for the tuned stacked ensemble falls to 0.431 under spatial-block validation, and a leave-one-region-out spectrum sends tree-ensemble skill toward zero
• LASSO holds out-of-sample R-squared 0.276 with half the county held out while XGBoost falls to 0.007, so the regularized linear model extrapolates best
• Double ML puts the direct coastal premium at +29.8% per square foot (95% CI [0.16, 0.33] log points) against a naive gap of 0.904 log points, with a +79.4% total-premium reading under structural controls only
• The naive private-school premium of 0.442 log points collapses to +0.7% (95% CI [-0.11, 0.11] log points) after adjustment for where private schools locate
• Every manuscript number regenerates offline from one cleaned table through generated macros, with tests and CI on the public repository

Methods

• Hedonic OLS (HC3-robust)
• LASSO sparsity check
• Nested-CV tuning and stacking
• Spatial-block cross-validation
• Double/Debiased ML
• Spatial block bootstrap
• Moran's I diagnostics