Matt Goldman

I am an empirical microeconomist and a Researcher in Microsoft's Chief Economist's Office. My research interests include Digital Markets, Behavioral Economics, and Theoretical Econometrics. At Microsoft, I work on the economics of sponsored search auctions, hardware pricing and using machine learning and data mining tools to automate economic models of causal inference as part of Microsoft Research's Project Alice Expedition. You can learn more about microeconomics research at Microsoft here.


Contact Information

mattgold at microsoft.com
99/4621
14820 NE 36th St
Redmond, WA 98052



Research (My Google Scholar Page)

Digital Markets

Position Auctions in Practice (with Justin Rao; Formerly known as: Experiments as Instruments: Heterogeneous position effects in sponsored search auctions )
Under Revision

The generalized second price (GSP) auction allocates billions of dollars of advertising via position auctions. Theory tells us that the GSP achieves the efficiency of the Vickrey-Clarke-Groves mechanism but with greater revenue, provided better positions increase click-through-rate by the same scaling factor for all ads. Since position is endogenous, this assumption is largely untested. We develop a novel method, "experiments-as-instruments," to re-purpose internal business experimentation to estimate the causal impact of position for 20,000 search ads. We strongly reject the multiplicatively-separable model, position effects differ by 100% across ads, which is partially explained by advertiser attributes.

Econometrics

Fractional order statistic approximation for nonparametric conditional quantile inference (with David M Kaplan)
Journal of Econometrics 196(2): 331-346
Supplemental Appendix Code for implementation of all methods is available at coauthor's website.

Using and extending fractional order statistic theory, we characterize the O(n-1) coverage probability error of the previously proposed (Hutson, 1999) confidence intervals for population quantiles using LL-statistics as endpoints. We derive an analytic expression for the n-1 term, which may be used to calibrate the nominal coverage level to get O(n-3/2[log(n)]) coverage error. Asymptotic power is shown to be optimal. Using kernel smoothing, we propose a related method for nonparametric inference on conditional quantiles. This new method compares favorably with asymptotic normality and bootstrap methods in theory and in simulations. Code is provided for both unconditional and conditional inference.

Nonparametric inference on (conditional) quantile differences and interquantile ranges, using L-statistics (with David M Kaplan)
The Econometrics Journal. doi:10.1111/ectj.12095
Supplemental Appendix Code for implementation of all methods is available at coauthor's website.

We provide novel, high-order accurate methods for nonparametric inference on quantile differences between two populations in both unconditional and conditional settings. These quantile differences corresponds to (conditional) quantile treatment effects under (conditional) independence of a binary treatment and potential outcomes. Our methods use the probability integral transform and a Dirichlet (rather than Gaussian) reference distribution to pick appropriate L-statistics as confidence interval endpoints, achieving high-order accuracy. Using a similar approach, we also propose confidence intervals/sets for 1) vectors of quantiles, 2) interquantile ranges, and 3) differences of linear combinations of quantiles. In the conditional setting, when smoothing over continuous covariates, optimal bandwidth and coverage probability rates are derived for all methods. Simulations show the new confidence intervals to have a favourable combination of robust accuracy and short length compared with existing approaches.

Comparing distributions by multiple testing across quantiles (with David M Kaplan)
Under Review
Code for implementation of all methods is available at coauthor's website.

When comparing two distributions, it is often helpful to learn at which quantiles there is a statistically significant difference. This provides more information than the binary "reject" or "do not reject" decision of a global goodness-of-fit test. Framing our question as multiple testing across the continuum of quantiles, we show that the Kolmogorov-Smirnov test (with appropriately modified interpretation) achieves strong control of the familywise error rate. However, its well-known flaw of low sensitivity in the tails remains. We provide an alternative method that retains such strong control of familywise error rate while also having even sensitivity, i.e., equal pointwise type I error rates at all quantiles across the distribution. Our method computes instantly, using our new formula that also instantly computes goodness-of-fit p-values and uniform confidence bands. To improve power, we also propose stepdown and pre-test procedures that maintain asymptotic familywise error rate control. One-sample (i.e., one known distribution, one unknown) and two-sample (i.e., two unknown distributions) cases are considered. Simulations, empirical examples, and code are provided.

Modeling Consumer Preferences and Price Sensitivities from Large-Scale Grocery Shopping Transaction Logs (with Mengting Wan, Di Wang, Matt Taddy, Justin Rao, Jie Liu, Dimitrios Lymberopoulous, and Julian McAuley)
Proceedings of 2017 International World Wide Web Conference (WWW'17), Perth, Australia

In order to match shoppers with desired products and provide personalized promotions, whether in online or offline shopping worlds, it is critical to model both consumer preferences and price sensitivities simultaneously. Personalized preferences have been thoroughly studied in the field of recommender systems, though price (and price sensitivity) has received relatively little attention. At the same time, price sensitivity has been richly explored in the area of economics, though typically not in the context of developing scalable, working systems to generate recommendations. In this study, we seek to bridge the gap between large-scale recommender systems and established consumer theories from economics, and propose a nested feature-based matrix factorization framework to model both preferences and price sensitivities. Quantitative and qualitative results indicate the proposed personalized, interpretable and scalable framework is capable of providing satisfying recommendations (on two datasets of grocery transactions) and can be applied to obtain economic insights into consumer behavior.

Behavioral Economics

Optimal Stopping in the NBA: Sequential Search and the Shot Clock (with Justin Rao)
Journal of Economic Behavior & Organization 136 (2017): 107-124.

We study how experienced agents solve a sequential search problem. In professional basketball teams must shoot within 24 seconds of the start of a "possession." The decision of when to shoot requires weighing the current shooting opportunity against the continuation value of a possession. At each second of the "shot clock," optimal play requires that a lineup's reservation shot value equals the continuation value. We empirically test this prediction with a structural stopping model. Most lineups adopt a reservation threshold that matches the continuation value closely. Overall, the lineups we study capture 84% of the gains of a dynamic vs. an optimal fixed threshold. Lineups with more shared playing experience performed better on average. Observed mistakes lean towards "impatience" - the adopted threshold is either in too low or has excess steepness - meanings too many shots are taken early in the possession.

Loss Aversion Around a Fixed Reference Point in Highly Experienced Agents (with Justin Rao)
Under Review
Press Coverage ESPN whiteboard animation Blog Posts: Economics in Action, Sloan Sports Analytics Blog

We study how reference dependence and loss aversion motivate highly experienced agents, professional basketball players. Loss aversion predicts losing motivates if the reference point is fixed and losing discourages if it adjusts quickly. We find a "losing motivates effect" so large that an average team scores like a league leader when trailing by ten points. Optical tracking of players' movements shows this effect comes through differential exertion of effort. Betting spreads and lagged score margin show that expectations do not influence the reference point, which is stable around zero, far less malleable than previously found in less experienced agents.

Holier than Thou? Testing Theories of Social Information in Charitable Giving using a Natural Field Experiment (with Jim Andreoni and Marta Maras)
Under Revision

We study a six-year campaign to raise funds to build a new church. Every Sunday, the priest announced donations, names, and addresses of donors, with surprise changes in the presentation of this social information. This unique data allows tests of hypotheses on how social information affects giving. We examine "fitting in" (neighborhood effects, norm conformance), and "standing out" (social-image, information signaling, conspicuous giving). Early in the campaign, we observe significant fitting-in. Over six years, however, the dominant effect of social information is to encourage standing-out. Moreover, information affects how social comparisons are formed, sometimes with unintended consequences.


Basketball Analytics

Optimal Strategy in Basketball (with Brian Skinner)

Press Coverage: Nylon Calculus

A chapter of Chapman & Hall/CRC Handbook of Statistical Methods for Design and Analysis in Sports

Live by the 3, Die by the 3: The Price of Risk in the NBA (with Justin Rao)

Press Coverage: The Wall Street Journal, Yahoo!
20 minute talk on this paper available on Presentations page

Proceedings of MIT Sloan Sports Analytics Conference 2013

Effort vs. Concentration: The Asymmetric Impact of Pressure on NBA Performance (with Justin Rao)

Press Coverage: Wall Street Journal (Weekend Edition 3/17/2012), Business Week, ESPN.com,
ESPN The Magazine (Paper awarded "ESPN Fan Choice" for best paper in the conference)

Proceedings of MIT Sloan Sports Analytics Conference 2012

Allocative and Dynamic Efficiency in NBA Decision Making (with Justin Rao)

Press Coverage: Wall Street Journal, ESPN.com, Slate
This is a preliminary version of ``Optimal Stopping in the NBA".

Proceedings of MIT Sloan Sports Analytics Conference 2011