Pablo Zavala · AI Safety Evaluation · Research Engineering
A Carnegie Mellon big-data coursework notebook that asks whether words in daily Reddit news headlines predict the Dow. Across 3,183 filtered words, marginal regressions with Benjamini-Hochberg FDR control leave exactly one return-predictive word, damn, a survivor the notebook reads as minimal predictive ability rather than discovery; volatility keeps 12 words and carries genuine persistence. Adding previous-day volatility lifts the lasso from zero selected words to an in-sample R-squared of 0.262, a compact lesson in multiple testing and confounding.
Across 3,183 headline words at 10 percent FDR, returns keep one survivor (damn, p = 1.0262e-05) versus 12 words for volatility; adding previous-day volatility lifts the lasso from zero selected words to in-sample R-squared 0.262.
In-sample evidence from one executed notebook on a dataset ending July 1, 2016; the source CSVs stay outside the public repo for license reasons, so a rerun starts from the Kaggle download plus the derived word files the README documents.
Role: Solo coursework: sparse text pipeline, marginal screening with FDR, lasso and double lasso, bootstrap, and interpretation.
| Axis | Sample | Evaluator | Result | Limitation |
|---|---|---|---|---|
| Primary output | One executed Jupyter notebook: 3,183 filtered words by 1,988 aligned trading days | Marginal OLS p-values, Benjamini-Hochberg FDR at q = 0.1, LassoCV with a time-series split, double lasso, bootstrap | Returns keep a single FDR survivor while volatility keeps 12 words; previous-day volatility carries most of the predictable signal. | In-sample fit on a single dataset that ends July 1, 2016. |
| Evidence level | Public repo with the executed notebook and README | Saved cell outputs and cell source, checkable without execution | Each public number traces to a printed output or committed cell source in the notebook. | The source CSVs stay external for license reasons; reruns need the Kaggle download plus two word files that arrive through the course rather than a public download. |
| Affiliation | Carnegie Mellon coursework | Notebook author line and file dates | Spring 2025 graduate coursework; filesystem dates place the notebook at February 26, 2025 and the figure exports at March 29, 2025. | Course handouts, prompts, and grading context stay private. |
| Page updated | July 2026 | Site content ledger | Entry drafted July 9, 2026 from the original executed notebook. | Numbers reflect the notebook's saved outputs rather than a fresh rerun. |
Two histograms carry the core finding. Word-level p-values for returns spread evenly, the signature of noise, while the volatility panel piles up near zero; Benjamini-Hochberg turns that visual contrast into counts, one survivor against twelve.
Every number quoted on this page appears in a saved cell output or the committed cell source of the notebook, so a reader can verify each claim by opening the file before running anything.
Open the committed notebook and check every quoted number against its saved outputs and cell source; that check runs on the file alone. A full rerun needs all four input CSVs: RedditNews.csv and DJIA.csv arrive via the README's Kaggle pointer, while WordsFinal.csv and WordFreqFinal.csv, the course-provided derived files the README describes, arrive through the course rather than a public download. One caution remains: the archived pass ran some cells out of order, so a clean rerun makes the fair test.
Screening thousands of headline words against market outcomes guarantees apparent winners by luck alone, so the analysis has to separate discipline-surviving signal from noise that slips through the gate.
The notebook pairs 1,989 trading days of DJIA prices, newest dated July 1, 2016, with daily Reddit world-news headlines, builds a sparse day-by-word count matrix, and filters 5,271 candidate words down to 3,183.
Marginal OLS regressions score each word twice, against daily returns and against log high-low volatility; Benjamini-Hochberg at q = 0.1 sets the discovery gate; LassoCV with a time-series split handles joint selection; a double lasso re-estimates volatility persistence against 1,429 selected word controls; a 30-resample bootstrap probes penalty stability.
Returns behaved like noise: 121 of 3,183 words landed under p = 0.05, the p-value histogram ran flat, and the FDR gate kept exactly one word, damn, at p = 1.0262e-05. Volatility kept 12 words, led by tunisia, georgia, and terror, yet previous-day volatility beat them all: the words-only lasso zeroed every coefficient, while adding the one autoregressive term produced six selected coefficients, in-sample R-squared 0.262, and a persistence coefficient of 0.442 that the double lasso trimmed to 0.313.
Everything stays in-sample on one dataset, the archived pass ran some cells out of order per the saved execution counts, and the notebook treats the exercise as feature-selection coursework rather than a tradable finding.
The public repository ships the executed notebook whose saved outputs and cell source contain every quoted number, plus a README pointing to the Kaggle source data, which stays external for license reasons.