Dharmateja P. Uddandarao Podcast Transcript
Dharmateja P. Uddandarao joins host Brian Thomas on The Digital Executive Podcast.
Brian Thomas: Welcome to The Digital Executive. Today’s guest is Dharmateja Uddandarao. Dharmateja Uddandarao is a distinguished data scientist and statistician working at the intersection of data science, statistics, and economics. He holds a master’s in analytics with a specialization in statistical modeling from Northeastern University and a bachelor’s in computer science and engineering with a specialization in data science from the National Institute of Technology Trichy.
He currently serves as senior data scientist and statistician at Amazon, and previously worked in senior roles for Capital One and Aetna. His work transforms complex data into measurable scientific and economic value for high-stakes business decisions. He is a fellow of the Institute of Analytics and the Royal Statistical Society, elected member of the International Statistical Institute, and holds the CStat award by RSS, PStat accreditation by American Statistical Association.
Brian Thomas: Well, good afternoon, Dharmateja. Welcome to the show.
Dharmateja Priyadarshi Uddandarao: Hey, thank you very much Brian. It’s my pleasure.
Brian Thomas: Absolutely, my friend. I appreciate it, especially you’re hailing out of the great state of Washington in Seattle. I’m in Kansas City, two-hour difference, but Washington, I, I spent some time there, and I just appreciate you making the time today.
So Dharmateja, let’s jump right into your first question. Your path spans computer science and data science at the National Institute of Technology in Trichy, a master’s in statist- statistical modeling at Northeastern, and senior roles at Aetna, Capital One, and now Amazon, plus a rare trifecta of accrediga- accreditations and election to the International Statistical Institute.
Brian Thomas: What drew you specifically to the intersection of data science, statistics, and economics, and how did that focus on causality become the through line of your career?
Dharmateja Priyadarshi Uddandarao: Yeah, the honest answer is that I didn’t set out to chase causality. Over time in my career I learned to build models, like ML models, and predict and infer not just, like, what will happen, but what actually is true and how confident can I be.
Then I kept hitting the same wall. We would build some beautiful high accuracy models ship a decision back of it, and some leaders would again come back and ask the same question every time, like, “But did our product or feature cause that, or would the change have would have happened anyways?”
And the traditional models couldn’t answer it. It was never built to or designed to actually. So, like, let me make like with a concrete and a small example. Let’s say we launched a new recommendation engine and the leadership looked at year-over-year revenue, and it was up about, like, fifteen percent year over year, let’s say.
Everyone wants credit for the feature, you know, but the sales rise up year over year anyways. Like the trend, marketing, seasonality, purchasing power, everything goes up every year. That’s a fifteen percent was everything plus like a kitchen sink. The real question was how much of it was due to the recommendation engine that we actually deployed, and how much would have anyway happened regardless?
That gap between what happened and what would have happened is the whole game and what sits exactly where the three fields overlap that I’m talking about. Like the statistics gives you the language of uncertainty. The data science gives you the ability to work at scale with messy and high-dimension data.
And economics or econometrics is one of the discipline that, you know, obsessed over centuries over inferring cause from data. Economists almost never get a lab, you know. They tease they tease out the causation out of the world as it actually unfolds. That is precisely, precisely the problem that most businesses have.
So causality became my through line because it’s the thing every stakeholder actually cares about, even when they ask for something else. Like a VP doesn’t really want a prediction. They want to know which lever to pull to make a product successful. So that’s the causal question, right? So prediction tells you what the world looks like.
Causal inference tells you what the world would look like if you acted differently. Businesses run on the second one. As of the acc-accreditations and the election of ISA, everything you are mentioning, the future generally belongs to the one who holds the both engineering muscle to build that skill like the products and everything, and the scientific discipline to know when a decision actually means what it means.
So that is where the whole, the, the path and the why causality is much more important.
Brian Thomas: Amazing. Thank you. And I just love this field. I’ve had some guests with a similar through line as yours on the podcast before, but I love the backstory. You didn’t really decide on ca- causality initially, but as you started working in these different models, you started to look at some of the cause and effects, and it really made you delve deeper.
And now you’re in that, I guess, that space where everything converges together, statistics, data science, and economics. And the thing that I thought was interesting is, you said pe- people say they want predictions, but they really wanna know what caused a certain thing, or if we did something different, at the end of the day, what is making things go up or down. So I really appreciate that.
Dharmateja Priyadarshi Uddandarao: That’s true.
Brian Thomas: You bet. Absolutely. Dharmateja, next question for you. Most of your articles frame causal economics as a new paradigm, bringing together classical economic- econometrics, machine learning, and causal inference theory. Why do you believe these three traditions need to be unified?
And what becomes possible when a data scientist can draw on all three rather than staying in just one camp?
Dharmateja Priyadarshi Uddandarao: Oh, yeah, because each of three traditions is brilliant at at excel– exactly the thing the other two are bad at. You know, and the most data scientists are trained to live with like one or two but all three of them are like required to be successful especially in the current business era.
So the classical economics that you are mentioning has the deepest theory of identification, like the formal conditions under which the number you compute actually equals the causal effect we care about. But it’s often rigid, you know, like the linear forms and the strong assumptions of how the variables relate.
But it’s a little far away from how business operates, I would say in terms of measurement. And the machine learning is like a mirror image. It, it is extraordinary at flexibility. It learns complicated, like non-linear or high dimension patterns with no hand-holding. But left alone, it’s actually causally blind, I would say, it optimizes for prediction, and the model can predict beautifully while being completely wrong about the cause. Like for example, it can happily tell you that ice cream sales increased drownings in summer, though they both are not related, but it’s not causing each other.
But that’s how the ML models they only care about the variables at which they are trained at. And the cause inference in theory, like the potential laid outcomes and counterfactuals, is the connective tissue, I would say. It gives you the language to even state what a causal effect is, the difference between the world where you acted and the parallel world where you didn’t.
Let’s discuss a small example, maybe of why you need all three at once. Like let’s see an ad measurement case. The question was let’s say whether seeing an ad caused users to convert. The name or ML style answer would be like exposed users like convert at like twenty-three percent, for example, and maybe unexposed users are like at twelve percent.
It’s like an eleven-point gap, right? Eleven percent. And we can say that’s the difference. But the ad, ad exposure isn’t generally random. The platform algorithm shows ads to most active users or based on the keywords and what they’re looking for. So the exposed group was already more likely to convert anyways.
That’s the confounding, right? So… And it’s an econometric problem, not an ML problem. So we use ML like a logistic model over age or engagement or like a device which they use to see the ad or whatever. We can use this to estimate each user’s propensity to be shown the ad and then build a matching comparison of genuinely similar users, which is causal inference design.
The harnessed bias-adjusted lift might come out to be like, let’s say, eight percent, not eleven. So ML found the structure econometrics keep us harnessed about the number actually meant. You need both. And the person who only had one tool would have over-reported the effect of ad, le-let’s say, by a quarter here, for example.
The field has figured out how to marry these all three formally. The clearest example is like double machine learning, where you see flexible ML to observe all the confounding structures, but wrap it up in an econometric framework that hands back you the clean, unbiased, statistically valid estimate. What becomes possible when one person holds three?
If that is your question, then we stop being limited to the tools and start being driven by important questions. Like a data scientist unconsciously reshapes every problem to fit into their hammer. The ML person predicts the and the economists generally regress over every, the same thing. But the person who holds all these three asks what actually true here and what’s the most regress way to find out, then reaches the right instrument.
Practically, that means estimating the real effect from observation data we already have and still being able to look like at a business leader in the eye and defend the number that we came up with. So this is where unifying all three, like econometrics, ML, and causal inference, are much more powerful in any business setting.
Brian Thomas: Amazing. Thank you for unpacking all that for our audience. You know, you talked about early on in the, in the question or your answer most data scientists, typically focus in one area. But you state that, that all three provide more accurate output and more possibilities there. And you shared that one example of the ads.
That ad, ad exposure just isn’t random. It’s about that a- audience engagement, and you covered causal inf- inference design, which I thought was amazing. So again, I appreciate it for myself and for our audience. Dharmateja- Of course … question three. A lot of your published work is about estimating the real impact of a program, feature, or investment using methods like propensity score matching and counterfactual forecasting.
When a company launches something new and sees the numbers move, how do you actually separate genuine cause from everything else happening at the same time? And where do teams most often fool themselves?
Dharmateja Priyadarshi Uddandarao: Yeah, actually separating genuine cause and noise is very important. This is the heart of it, and the honest truth is that the numbers moving is almost never evidence of anything by itself When a company ships something and the metrics goes up, there are actually like five or ten other explanations sitting in the room.
You know, like our job is to kill them one by one. The core problem has a name, you know, like the counterfactual. When the revenue rises five percent after a launch, for example, the number you actually care about is unobservable. What would revenue have done if we hadn’t launched it? You never get to see the other side of the world.
So all the causal inference is the science of constructing a credible stand-in for that missing world. Let me give you like couple of examples or a couple of concrete playbooks from my own research work. First, like in a counterfactual forecasting, like on the recommendation engine example that we discussing, we couldn’t run an AB test let’s say.
And the feature went up to everyone, and the leadership wanted answer now, not like in three months or one year. So we can build a control out of the matched users. We can train a time series model on only the pre-launch data, week zero to, let’s say, week eighty or something before the launch, so it had no lo– no knowledge of the launch, and then project it forward.
That forecast is a counterfactual. What revenue would have done if nothing has changed? The gap would be like between the actual and the forecast in effect is what we are looking for. The launch week or the actual was, let’s say, fifteen hundred dollar, for example. And like a counterfactual is thirteen hundred.
It’s roughly two hundred dollars lift, right? So by October, let’s say it is a three hundred dollar per week, and by year-end it is, let’s say, five hundred dollar per week. Cumulatively, a few million dollars and critically the gap gr-grows over time, by which we can infer, infer that it wasn’t just one week novelty bump, but a durable behavioral change.
And the second one that we were discussing is the propensity score matching. Like, for example, in the ads case I was mentioning before. There I can match each, each exposed user to a genuinely similar unexposed user on propensity score. The naive gap is eleven point shift to a credible eight point.
And there is a discipline, you know. Even after matching one covariate, let’s say device level engagement, still had a st-standardized difference of more than point one. Threshold is generally try to hold. So it can be flagged as a double adjustment. That is it. Decidual is exactly the kind of thing, teams sweep under the rug That residual imbalance problem is actually what pushed me to develop something I call like pre– metric pre-balancing or target metric pre-balancing.
The standard move in the propensity score matching is to balance covariates like age or devices, but often it is still unbalanced. The baseline value outcome metric itself is important to be treated. And before you even touch it, the standard PSM can completely miss that still in your hand, and confidently it can give a wrong number.
So I generally pre-balance explicitly on pretreatment trajectory of any target metric, not just surrounding covariance, before I even compute a lift change in a metric. In one of the articles, I was discussing the same thing on enterprise infrastructure too, actually. When I was benchmarking the data pipelines across changes in data architecture or tech stacks the pre-balancing causal framework I developed even it evens out like pretreatment conditions like workload size or processing complexity and something else which is very important for the tech deployment before even comparing the pipeline configuration.
So fifteen percent drop that we generally see in execution time isn’t like an accidentally created to the whole migration, but it can really be a quiet day of traffic. You know, so where people can fool themselves is like multiple drags, like confounding or concurrent events. Confounding is something that didn’t actually account for driving both actions and outcomes, I would say.
And the concurrent events is like you launch the same week multiple events and the metric moved, not just because of the one event that we are claiming for. So the discipline is most flexible. Before we believe in lift or before I believe a lift, I try to kill it, like stare at the counterfactuals, specifically list every alternative, check, check the groups were comparable before the intervention and treat a result leadership is hoping for with more suspicion, not less.
Because the goal isn’t to be the person who proves the launch worked. It’s to be the person that organization trusts because you will tell them when it didn’t work out as well. That’s the– that is where most of the work relies on, you know, separating genuine cause from the noise.
Brian Thomas: Thank you. Appreciate that. There’s a lot to unpack there. But at the end of the day, you know, you did really really d-delve into estimating the real impact of a program or a feature using those methods you talked about, propensity score matching and counterfactual forecasting. But you did… The thing I took away is the heart of this is really separating genuine cause from everything else, and I thought that was interesting. And Dharmateja, last question of the day, if you could briefly share, you’ve talked about organizations shifting from descriptive analytics to genuine decision intelligence. As AI and large scale models become more capable, where do you see causal inference and evidence-based economic reasoning heading over the next five, maybe ten years?
And what has to happen for businesses to trust AI systems not just to predict, but to actually guide high-stakes decisions?
Dharmateja Priyadarshi Uddandarao: Oh, yeah. We are at like at an inflection point, I would frame it. So the last decade of AI was like about prediction become– becoming cheap. The next decade is probably about decisions becoming more intelligent.
And we can’t get that from one the other without causality, I would say. Here is why. Like predict to AI, including the large language models everyone is excited, answer what is likely. But every genuine high-stake decision is a causal and econometric question. What happens if I act and is it worth it?
A model can predict churn beautifully and still not tell which intervention will reduce it or whether the intervention costs more to the customer than it’s worth. So prediction describes the world, but decisions require you to intervene in it. Technically, the trend is encouraging, and it’s where my most work is pointed as well.
The ML and the causal models or methods like double machine learning or causal forest is maturing out of… just out of research papers into, like, enterprise toolings. And the newest frontier is combining the causal structure with all the generative models, I would say. You know, the same spirit as the deterministic causal scenarios or analysis framework.
Like moving forward from a single forecast to a set of causal scenarios, that’s a direction. You know, the AI that reasons about alternate worlds, not just auto-completes this present one. But this is an important part. Trust won’t be solved by making models bigger, I would say. For a business to trust AI to guide high-stake decisions some things have to happen, like causal grounding.
A system guiding a decision must rest on estimates of effects with explicitly assumptions, not just correlation basis. And harnessed uncertainty, like it trusts what the system says, here is the effect, here is the confidence interval, and here is what I’m guessing. Like overconfident points estimates are sometimes like how we get into expensive mistakes, especially in economics and finances.
And economic context is very much important. The system has to understand the cost structure of the decision because the right action is always the one where the causal benefits outweigh the cost, or how much of a beautiful model it can be. So my prediction is that organizations that win won’t have the biggest models.
They’ll be the ones that pair AI scale with like causal and statistical rigor. Like treating AI as an instrument for evidence-based decisions with humans and economics firmly in the loop. The buzzword might fade over time, the discipline won’t. You know, knowing why, not just what, is what lasts forever.
Brian Thomas: Thank you. Really appreciate that. There’s a lot there to unpack, of course. And just kinda quickly highlighting some things. You talked about, current prediction AI, how, how cheap is AI going to be and the future or next iteration of that prediction will be, AI, how much more AI will be intelligent.
And you really went into each of those areas, but it’s important to learn here that businesses need to trust AI systems, again, not just to predict, but to really guide those decisions that are so, so important. And again, there’s just a lot into this, and I really love this topic with, with causal inference.
It’s just amazing. I, I appreciate that. And- Yep … Dharmateja, it was such a pleasure having you on today, and I look forward to speaking with you real soon.
Dharmateja Priyadarshi Uddandarao: Thank you, Brent. Thank you for having me. It’s my pleasure. Thank you very much.
Brian Thomas: Bye for now.
Dharmateja P. Uddandarao Podcast Transcript. Listen to the audio on the guest’s Podcast Page.
