Matt Goldman

I am an empirical microeconomist and a Researcher in the Economics and Computation Group at Facebook's Core Data Science. My research interests include Digital Markets, Behavioral Economics, and Theoretical Econometrics. At Facebook, I work on the econometrics of online experimentation. You can learn more about economics research at Facebook here.

Contact Information

mattgoldman5850 at

Research (My Google Scholar Page)

Digital Markets

Position Auctions in Practice (with Justin Rao; Formerly known as: Experiments as Instruments: Heterogeneous position effects in sponsored search auctions )

The generalized second price (GSP) auction allocates billions of dollars of advertising via position auctions. Theory tells us that the GSP achieves the efficiency of the Vickrey-Clarke-Groves mechanism but with greater revenue, provided better positions increase click-through-rate by the same scaling factor for all ads. Since position is endogenous, this assumption is largely untested. We develop a novel method, "experiments-as-instruments," to re-purpose internal business experimentation to estimate the causal impact of position for 20,000 search ads. We strongly reject the multiplicatively-separable model, position effects differ by 100% across ads, which is partially explained by advertiser attributes.


Orthogonal Machine Learning for Demand Estimation: High Dimensional Causal Inference in Dynamic Panels (with Victor Chernozhukov, Vira Semenova, and Matt Taddy)

We provide estimation and inference tools for a high-dimensional sparse treatment effect in a partially linear model in a panel setting. First, we estimate the expectation of treatment and outcome variables conditionally on lagged realizations of treatment, outcome and any other exogenous predictors using modern machine learning (ML) methods. Second, we construct a point estimate of the treatment effect using Lasso with the dependent variable and the covariates equal to the residuals from the previous regressions, respectively. We construct confidence intervals for single and many coefficients using a version of debiased Lasso. We apply these methods to estimation of heterogeneous demand elasticities -- a challenge frequently faced while optimizing pricing for a wide catalog of products -- and validate our estimates using experimental variation in prices.

Comparing distributions by multiple testing across quantiles (with David M Kaplan)
Journal of Econometrics
Code for implementation of all methods is available at coauthor's website.

We first show that one-sample and two-sample Kolmogorov-Smirnov tests may be interpreted as multiple testing procedures, nonparametrically testing equality at each point in the distribution with strong control of the finite-sample familywise error rate. Second, we provide an alternative procedure that distributes power across the distribution more evenly than the Kolmogorov-Smirnov test, which suffers low sensitivity to tail deviations. Third, we provide a formula for near-instant one-sample computation. Fourth, we improve power with stepdown and pre-test procedures. Finally, we extend our results to conditional distributions and regression discontinuity designs. Simulations, empirical examples, and code are provided.

Fractional order statistic approximation for nonparametric conditional quantile inference (with David M Kaplan)
Journal of Econometrics 196(2): 331-346
Supplemental Appendix Code for implementation of all methods is available at coauthor's website.

Using and extending fractional order statistic theory, we characterize the O(n-1) coverage probability error of the previously proposed (Hutson, 1999) confidence intervals for population quantiles using LL-statistics as endpoints. We derive an analytic expression for the n-1 term, which may be used to calibrate the nominal coverage level to get O(n-3/2[log(n)]) coverage error. Asymptotic power is shown to be optimal. Using kernel smoothing, we propose a related method for nonparametric inference on conditional quantiles. This new method compares favorably with asymptotic normality and bootstrap methods in theory and in simulations. Code is provided for both unconditional and conditional inference.

Nonparametric inference on (conditional) quantile differences and interquantile ranges, using L-statistics (with David M Kaplan)
The Econometrics Journal. doi:10.1111/ectj.12095
Supplemental Appendix Code for implementation of all methods is available at coauthor's website.

We provide novel, high-order accurate methods for nonparametric inference on quantile differences between two populations in both unconditional and conditional settings. These quantile differences corresponds to (conditional) quantile treatment effects under (conditional) independence of a binary treatment and potential outcomes. Our methods use the probability integral transform and a Dirichlet (rather than Gaussian) reference distribution to pick appropriate L-statistics as confidence interval endpoints, achieving high-order accuracy. Using a similar approach, we also propose confidence intervals/sets for 1) vectors of quantiles, 2) interquantile ranges, and 3) differences of linear combinations of quantiles. In the conditional setting, when smoothing over continuous covariates, optimal bandwidth and coverage probability rates are derived for all methods. Simulations show the new confidence intervals to have a favourable combination of robust accuracy and short length compared with existing approaches.

Modeling Consumer Preferences and Price Sensitivities from Large-Scale Grocery Shopping Transaction Logs (with Mengting Wan, Di Wang, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulous, and Julian McAuley)
Proceedings of 2017 International World Wide Web Conference (WWW'17), Perth, Australia

In order to match shoppers with desired products and provide personalized promotions, whether in online or offline shopping worlds, it is critical to model both consumer preferences and price sensitivities simultaneously. Personalized preferences have been thoroughly studied in the field of recommender systems, though price (and price sensitivity) has received relatively little attention. At the same time, price sensitivity has been richly explored in the area of economics, though typically not in the context of developing scalable, working systems to generate recommendations. In this study, we seek to bridge the gap between large-scale recommender systems and established consumer theories from economics, and propose a nested feature-based matrix factorization framework to model both preferences and price sensitivities. Quantitative and qualitative results indicate the proposed personalized, interpretable and scalable framework is capable of providing satisfying recommendations (on two datasets of grocery transactions) and can be applied to obtain economic insights into consumer behavior.

Behavioral Economics

Optimal Stopping in the NBA: Sequential Search and the Shot Clock (with Justin Rao)
Journal of Economic Behavior & Organization 136 (2017): 107-124.

We study how experienced agents solve a sequential search problem. In professional basketball teams must shoot within 24 seconds of the start of a "possession." The decision of when to shoot requires weighing the current shooting opportunity against the continuation value of a possession. At each second of the "shot clock," optimal play requires that a lineup's reservation shot value equals the continuation value. We empirically test this prediction with a structural stopping model. Most lineups adopt a reservation threshold that matches the continuation value closely. Overall, the lineups we study capture 84% of the gains of a dynamic vs. an optimal fixed threshold. Lineups with more shared playing experience performed better on average. Observed mistakes lean towards "impatience" - the adopted threshold is either in too low or has excess steepness - meanings too many shots are taken early in the possession.

Loss Aversion Around a Fixed Reference Point in Highly Experienced Agents (with Justin Rao)
Press Coverage ESPN whiteboard animation Blog Posts: Economics in Action, Sloan Sports Analytics Blog

We study how reference dependence and loss aversion motivate highly experienced agents, professional basketball players. Loss aversion predicts losing motivates if the reference point is fixed and losing discourages if it adjusts quickly. We find a "losing motivates effect" so large that an average team scores like a league leader when trailing by ten points. Optical tracking of players' movements shows this effect comes through differential exertion of effort. Betting spreads and lagged score margin show that expectations do not influence the reference point, which is stable around zero, far less malleable than previously found in less experienced agents.

Holier than Thou? Testing Theories of Social Information in Charitable Giving using a Natural Field Experiment (with Jim Andreoni and Marta Maras)

We study a six-year campaign to raise funds to build a new church. Every Sunday, the priest announced donations, names, and addresses of donors, with surprise changes in the presentation of this social information. This unique data allows tests of hypotheses on how social information affects giving. We examine "fitting in" (neighborhood effects, norm conformance), and "standing out" (social-image, information signaling, conspicuous giving). Early in the campaign, we observe significant fitting-in. Over six years, however, the dominant effect of social information is to encourage standing-out. Moreover, information affects how social comparisons are formed, sometimes with unintended consequences.

Basketball Analytics

Optimal Strategy in Basketball (with Brian Skinner)

Press Coverage: Nylon Calculus

A chapter of Chapman & Hall/CRC Handbook of Statistical Methods for Design and Analysis in Sports

Live by the 3, Die by the 3: The Price of Risk in the NBA (with Justin Rao)

Press Coverage: The Wall Street Journal, Yahoo!
20 minute talk on this paper available on Presentations page

Proceedings of MIT Sloan Sports Analytics Conference 2013

Effort vs. Concentration: The Asymmetric Impact of Pressure on NBA Performance (with Justin Rao)

Press Coverage: Wall Street Journal (Weekend Edition 3/17/2012), Business Week,,
ESPN The Magazine (Paper awarded "ESPN Fan Choice" for best paper in the conference)

Proceedings of MIT Sloan Sports Analytics Conference 2012

Allocative and Dynamic Efficiency in NBA Decision Making (with Justin Rao)

Press Coverage: Wall Street Journal,, Slate
This is a preliminary version of ``Optimal Stopping in the NBA".

Proceedings of MIT Sloan Sports Analytics Conference 2011