Putting Models into Practice

This past summer, Raghav Singal joined the Tuck faculty after earning his Ph.D. in Operations Research in 2020 and spending a year as a data scientist intern at Amazon. Ever since his undergraduate days at the University of Toronto, Singal, who played tennis at a national level as a junior, has been interested in the intersection of machine learning and using data to help us make better decisions.

That inclination first manifested for Singal during his 14-month internship in finance while he was a student in Toronto. He spent a lot of that time doing portfolio optimization, which is the practice of using data to decide how much money to invest in certain assets when forming a portfolio. Singal had also done some scientific research at Toronto, and when he compared it to his time in finance, he found that the process of gaining new knowledge was more intellectually stimulating than putting old knowledge into practice, and he liked the intellectual freedom of academia.  

Singal pursued his doctorate at Columbia, and through professor Martin Haugh, started thinking more about fantasy sports—online games using virtual teams composed of real players, whose performance is based on statistics from their actual performance. “Given my background in portfolio optimization, fantasy sports just clicked,” he says. “It’s a very similar concept: your assets, instead of being stocks or bonds, are athletes in the real world, and you have historical data on them.” The major difference is in the definition of winning. With financial assets, you don’t have an opponent, so it’s all about absolute performance. In fantasy sports, however, you only have to perform well enough to beat your competition. So Singal proposed a model that adapts your athlete portfolio as your opponents form their own portfolio. He and Haugh then leveraged that mathematical model to design tractable computational algorithms, which they used to invest money in fantasy football during the 2017 NFL season. They ended up tripling their money in just three months and donated their winnings to charity. The paper that flowed from their research—“How to Play Fantasy Sports Strategically (and Win)”—was published in Management Science in 2020.

With another group of advisors at Columbia, Singal began studying online advertising. But before he started building models, he interned at Adobe, in San Jose, CA. Most people know Adobe from their suite of creative software such as Photoshop, Illustrator, and InDesign, but the company also has a huge presence in digital marketing management, through its Adobe Advertising Cloud software. The software helps businesses decide how and where to advertise, and understand which ads are working. In this space, Singal learned about the problem of attribution—figuring out which marketing channels (emails, social media, search engines) bore the most responsibility for a particular sale. Practitioners have formed heuristics to guess at attribution, giving prominent weight to either the first ad a consumer sees, the last ad they see, or applying uniform weight to all the ads they see. Starting from those heuristics, Singal and his colleagues at Columbia created a model that served as a unifying framework for all such heuristics and allowed the researchers to see each one’s strengths and limitations. Then, they used the resulting insights to propose a more justified approach to attribution. Interestingly, their mathematical modeling led them to advocate for the uniform heuristic, but one that accounts for the possibility that some of the ads a consumer sees may have been worthless. Their model is the basis of a paper published online last month by Management Science, “Shapley Meets Uniform: An Axiomatic Framework for Attribution in Online Advertising.”  

Once Singal completed his Ph.D., he approached the attribution problem from a different angle: as a supply chain challenge for Amazon. Similar to how advertisers want to attribute particular sales to certain marketing activities, Amazon wanted to attribute errors from its supply chain forecasts to particular links in its network of models. “Big data-driven organizations have maybe 100 models running their operations,” Singal explains. “Each model has an output, which becomes the input of another model. If something unexpected happens—say, inventory is really low—it’s hard to understand if the data was bad, or if the model messed up.” Singal and his colleagues at the University of Florida and Amazon developed a model to better understand how information flows in these types of networks, and published their work with the International Conference on Machine Learning (ICML).  

Singal is co-teaching the core course Analytics I with Professor Jim Smith and is looking forward to developing an elective that taps into his passion for machine learning and data-driven modeling.