Your Marketing Measurement Is Broken: Here’s How to Fix It

Ryan McClurkin:
I say marketing measurement is broken because of the way operators try to solve the problem. they aren’t given a full tool set. And really, if you’re trying to solve the problem and you’re trying to scale your ad budgets or you’re trying to create efficiency in ad budgets, you really don’t understand the problem until you lived it as an operator. And That’s one of the biggest takeaways. Is until you’ve lived it as an operator, you don’t realize what actually happens in the war room, I see three core flaws with traditional measurement. And it’s still how I would say 90 to 95% of businesses measure marketing. And there’s there’s really three core flaws. The first one is

Zach Thompson:
Hey, welcome back to The Marketer Exchange. I’m Zach Thompson, and I’m excited to be back in the Haslam College of Business, and really excited today to have Ryan McClurkin, who is an investor and co-founder of Insighta. Ryan, great to have you.

Ryan McClurkin:
have Yes, I’m happy to be here and be in the Haslam College of Business again as well. I’m one of the alumni. So happy to be here and chat with you.

Zach Thompson:
It’s great to have you back. Thanks so much for joining us. You kind of got your start in mathematics and economics and have now ended up in this wild world of marketing and analytics. Tell us a little bit about that journey.

Ryan McClurkin:
Yeah, absolutely. Yeah. I w and I was undergrad I was a double major in math and economics. always had a proclivity to kind of applied statistics and applied economics. from there I went, I got a graduate degree in applied economics. and straight from there I went into the re retail analytics. at the time, you know, it was about 2010-2011, is when you really started to see retailers take on the analytics. They finally had the compute power to do it. so it was right before you started to have business analytics degrees and data science degrees. the closest thing to it was applied econometrics, which is what my specialty was. went to work for a local company here in Knoxville. kind of a mid-size enterprise company and started to lead their advanced analytics team.

So built their advanced analytics team a little bit from the ground up. we didn’t really have one at the time, not a lot of companies had one. It was just starting to get into that phase. ended up becoming their chief analytics officer and I did that role for roughly fifteen years. but I had a lot of exposure in that business to every area of retail analytics. Okay, so merchandising analytics. TV, it was a is a TV company, so a lot of television analytics. There’s a niche in that world, a lot of customer analytics, and you know, through from 2014 through 16, we really put a lot of effort into growing our e-commerce entity, and that got some expertise in the advanced marketing analytics.

So I started to see a lot of problems there, started to discover that a lot of s companies had a similar problem, right, and how they measure. And you know, that kinda led me further on my path to where I’m at today, but just started to gain some experience in that space and realized that a lot of people had that a similar problem. So, you know, started to wanna solve that for more than just, you know, one company and took that experience and knowledge and Started to build Insighta. But yeah, developed a really advanced specialty in a lot of areas of analytics. But the one that I saw the biggest need in the space for was solving the performance marketing analytics the right way.

There are just so many flaws with how operators try to solve you that issue and they don’t have the right solutions. So that’s what led to the ideation behind Insighta.

Zach Thompson:
So you’ve talked about the problems that you saw within kind of being on the client side, on the brand side, some of the problems that you saw within marketing. I’ve heard you say marketing measurement is broken. So

Ryan McClurkin:
Yes.

Zach Thompson:
tell us a little bit more about that. Like what is the problem that you’ve identified on that kind of brand client side?

Ryan McClurkin:
Yeah, I think what happens is, and I say marketing measurement is broken because of the way operators try to solve the problem. So they aren’t they aren’t given a full tool set. And really, if you’re trying to solve the problem and you’re trying to scale your ad budgets or you’re trying to create efficiency in ad budgets, you really you really don’t understand the problem until you lived it as an operator. And that’s the biggest that’s one of the biggest things. That’s one of the biggest takeaways. Is until you’ve lived it as an operator, you don’t realize what actually happens in the war room, right? And I see three core flaws with traditional measurement. And it’s still how I would say 90 to 95% of businesses measure marketing. And there’s there’s really three core flaws.

The first one is how you are actually allocating the advertising dollars relative to the benefit that gets created. And simplistic dashboards, it’s notorious for it. You’ll pull up a dashboard and you’ll say, hey, what did I spend in the last two weeks and what did I get in the last two weeks? And the reality of advertising is that you’re buying advertising today for a future benefit. Not today. Very few, very little of it actually impact you today. And yet advertisers are stuck in marketers are stuck in this world where they’re measuring ad spend today and benefit today and they’re not connected. So you have like these whiplash effects internally in the org. So you know, say economic times get hard and you’re decreasing your ad spend.

You have to get you know, you get constraints for finance and they say, Hey, you gotta drop your ad budget by twenty-five percent. In a traditional marketing world, you’ll just see your ROAS rise by twenty-five percent because the next week you have this hyper focus on what is it doing to our revenue, right? And return on ad spend, ROAS, right? And what happens is your revenue was bought under a higher ad spend, but then you come in and you’re measuring 25% less ad spend and the revenue stays the same, right? And so you see this jump in your efficiency metrics, but it’s really a false positive, but it creates a terrible narrative through the org. Because then the or then the narrative becomes, well that advertising doesn’t work.

Look, we cut it, revenue stayed the same, but then 13 weeks later, revenue starts to fall and everybody’s wondering what’s going on. Right. And it’s because you’re not really connecting the ad spend to the benefit that it actually creates. Right. That’s one of the biggest problems. when you’re testing new strategies, it’s also a big problem. Like say you turn on an ad spend for 50,000 a week and you try to measure that benefit in the first week. If you measure it against 50,000 an ad spend, it’ll look terrible. because that 50,000 is buying you future periods of benefit, not that single period. So that’s one of the f very first battles that, you know, I started to see.

I talked to other operators and it was like the one of the hardest battles they face with their you know, finance teams and budgeting and accounting teams is that right there, right? And the problem is that a CFO might not be in tune with it that way. Marketers know that happens, but the thing they’ve got to put on a piece of paper and put in front of their org says that. Yet they try to explain their way out of it. That’s one of the biggest problems. The second biggest problem that I see in the space is the reliance on click-based attribution, purely click-based. you know, there’s some statistics out there, I think between 95 to 97% of marketing impressions are never even clicks. That’s like a very small sliver of the actual marketing impressions that someone gets.

And so what happens in that space is when you when you have this reliance on click-based attribution, you just don’t see the full picture. and I and I call it less addressable media. Things like connected TV, things like streaming audio, right? billboards, linear TV, you name it, right? Any of the less addressable media, programmatic advertising, right, it generates a high level of view-through impressions that never show up in a click-based dashboard. Right? you And so what happens is these operators, they’re trying to make decisions. Every company wants to be more data oriented. Every company in America, every company in the world million. wants to be more data oriented and make decisions with data. Yet they have a fraction of the visibility that they need.

And so what happens is you’re you’re trying to make these decisions on click-based advertising, which that tends to rely heavily on the Google and Facebook world, meta world, right? And they dump all their money into those channels because they can measure the impact of those channels in their standard dashboards. But they really struggle to see, for example, the impact of CTV. And it’s a real impact, or direct mail. Direct mail is a fantastic channel. but you can’t measure it properly because you don’t have when it hit in-house and you can’t see it in that journey. And I call it sort of putting together a more full picture of the customer journey. Can you capture both clicks and impressions to really see what’s happening?

Because when you get into brand and upper funnel advertising, a lot of it is view-through-based, right? Or direct mailpiece hits or streaming audio hits, right? And can you see how that’s playing in that journey and have a more complete picture? That’s the second biggest area. The third biggest area, and this is especially true in the retail space, is that the real hedgehog for any retail company is lifetime value. you And a lot of companies don’t even have an LTV model. predictive lifetime value model. And I don’t mean lifetime value from a historical perspective. I mean the predicted value, the future value of your consumers is really what you should be optimizing on. Because that’s your lifeblood in the future, right?

And can you understand how your marketing tactics are impacting the future value? you wouldn’t believe how many tactics I see That generate very mediocre immediate benefit or immediate return on ad spend, but very good future value-based return on ad spend. And you’ll have companies, it’s very common for marketers to have, I’ll call them new customer targeted tactics, right? That they’re pure KPIs just to generate new customers and they’re not even, they’re not even targeting existing customers, which is a cheaper order to generate. That immediate return falls. And it’s generating a high amount of future value, but they’re blind to it. And so what they do is they tend to move money out of that because the immediate return is falling. And what are they trying to optimize on? Right?

You think about your optimization metrics, it’s really should be the lifetime value of the consumer. Because if you were doing that, you might invest differently. And you know, it’s one of those things I always give an example with a with the CFO. You know, say you were generating. a ROAS of three dollars, a return ad spend of three dollars, and you know your cost of acquisition was thirty dollars for a you know depending on what you’re selling and your margins that might or may or may not be good enough. But if you knew that consumer was gonna spend two hundred dollars with you in the next year, would you make the investment? you And almost every single time the answer is yes. But do you have visibility? Right?

So I think creating visibility from a lifetime value perspective, especially in the retail space, is incredibly important.

Zach Thompson:
What were some of the workarounds that you found on the client side before you got to like the solutions that you’re at now and that you’re building now? What were some of the because that relationship between marketing and finance can be quite tricky, right? Where yes, they want they want to see an immediate return for your activity for what you did today. But what were some of the workarounds?

Ryan McClurkin:
You the workarounds are difficult. You see people piece a lot of things together, right? So what you’ll see is marketers will piece together a GA4 dashboard or an Adobe Analytics dashboard, and then you know, for direct mail they might try to do a holdout test group or something like that to see, well what’s the impact. But the issue that you get is if you can’t put together a holistic picture. It becomes really difficult. Those conversations with finance become really difficult, right? Because there’s always this doubt of, well, how incremental is it, right? Or is it really touching, are they touching the same consumers, right? But you’re just counting the full benefit over here and it should be partial. Right.

And so I think until you have a holistic methodology, you see people piece a lot of things together. And that’s never a great path, right? If you really want to create, you know, a synergy with finance as a marketer, it has to be really open book, but you have to have a holistic picture. You know, and one of the other things I see too, and it’s it’s it seems trivial, but a lot of the marketing platforms are also what I call pixel-based revenue. So, you know, the way they’re tracking their success is they have this tracker on your website, for example, and when a conversion hits they see it and they have a value and that gets recorded in their platform, right? That revenue never matches what your actual revenue is as a company.

Because it cancels or the pixel may fire twice if someone refreshes the page, right? And so what happens is the other the other like, I’ll call it, you know, one of the big conflicts between marketing and finance is that a lot of marketers are using a benefit number, a revenue number, an immediate return number that doesn’t actually match what the company is making. And you know, one of one of my biggest you One of the things I’m most stickler on is actually using the conversion as source of truth. this is how much benefit we generated, whether it’s revenue, orders, leads, new customers, but like coming from the conversion source, not pixel-based, right? And so what I think you see happen is you see a lot of piecing it together, right?

And they’re trying the you know, marketers are trying to put these pieces together in a certain way that makes sense, but at the end of the day, it becomes insufficient for a finance team to re they’re they’re they’re cutting dry, right? And it becomes insufficient. And you know, that’s that’s on the immediate side. I think I see orgs go through a learning phase with lifetime value. Right? You have to first have an LTV model, right? Most organizations don’t even have one. So you have to be able to produce a predicted lifetime value model and then it’s a learning phase. But financiers are smart enough, finance teams, they know that’s the future. Now there’s always this pressure for immediate. immediate. But unless you take care of the future, you’ll never you’ll never hit the immediate.

That’s part of the issue. And they know that, right? But I think you have to give them a path to get there, or else it just be this hyper focused on immediate. If they don’t understand how much that customer’s worth or that these channels are actually generating this much future value and you can’t put it on a piece of paper, that becomes a really difficult conversation. They know though. They’re smart enough to know you can’t cut off your new customer stream and expect growth in the future. They know that. That’s why you see you but you’ll see companies focus on CAC, cost of acquisition, or cost per acquisition, CPA, right? You see them focus on that and they want certain numbers, that’s because they know that customer’s worth a certain amount.

But really putting it on a piece of paper and connecting the dots becomes a huge enabler for those conversations to happen.

Zach Thompson:
All right, so piecing it together is not the answer. You’re going to end up with marketing and finance with different factors

Ryan McClurkin:
That’s right.

Zach Thompson:
at the bottom line, right? So what do you see as the solution to deal with this problem that you’ve?

Ryan McClurkin:
I just do it over years. Yeah, and I think that’s where, you know, just the knowledge that I’ve gained and some of the experience I’ve gained is piecing together what I call I’m gonna call it a marketing data warehouse. But you know, in today’s world that’s it looks a little different, especially in the last two to three years with AI and we’ll talk more about that, but I think you need to capture three primary sources of data to even start to have this conversation. So inside of a unified data warehouse or unified data environment, you need data flowing of all the money you’re spending, everywhere you’re spending money. We call it ad platform data, right? But think about advertising platform data, that’s Google API feeds, Facebook API feeds, right?

Anywhere you’re spending money, StackAdapt, Criteo, Pinterest, TikTok, right? That all needs to be flown into a data warehouse so that I can do some modeling on it. Right. And in reality too, I see a lot of inefficiency in orgs where they have people, they’ll literally have a human going and downloading those dashboards to piece them together in Excel. And there’s far better ways from a data capture perspective. So that’s one vertical. The second vertical is you’ve got to start capturing all of the customer touch points. That’s the way I think about them. So the most simplistic one are website events. So you go to a website, you’re clicking around, you need visibility to that. Right? If you click through an ad, you’ll see it.

So there’s something called tagging on URLs that the event trackers capture. So we capture all of the customer events happening on the website. That’s that’s called first party data. So you put a you put a pixel or a tag on the website and it’s capturing all of the first party events. data. Okay. Now the question becomes: how do I move beyond? That click, right? Those are all clicks. You actually have to go to the website, you have to click around, you have to come from an ad, right? That’s click-based. That’s a fraction of the actual benefit, right? So how do you start to exit into impression-based, right? And this is where you have to work with your advertiser, okay? So if you’re if you’re doing advertising on direct mail or you’re doing advertising on connected TV, CTV, right?

The advertiser knows who’s watching. They know Zach is watching, or they know they delivered Zach a postcard in the in the in the mailbox that day. So one of the things that we specialize in and we do is we actually go and connect all those third-party data feeds. So those are third-party customer touch points. You have Okay, so you first party that I collect come into my website, and then the advertising company can tell you, I advertised to Zach 20 days ago. I put it, I put a mailbox, a mail. piece in his mailbox 10 days ago. Right. And what we do is we couple those. So we have an identity graph, we know that they belong to the same customer, and we start to piece together a more complete picture of what that journey looks like.

So those are called, we call them kind of customer event touch points, but think about them as any type of touch point you have with the consumer, you’re trying to capture any touch point. And you’re not gonna get all of them. That’s the misnomer, right? I’ll never tell you, you’re never gonna have 100% visibility. Like Google will never give you the impressions. They don’t even track it. Google Tech, they don’t even track it. So you’re never gonna have all of it, but can you be more complete than you are today? That’s what I say. Right? And I come from the operator perspective, I know it’s not complete. And I’ll never tell you it is, but can you be more informed than you are today? Absolutely. The third vertical is what we call the conversion source of truth.

So this is now connecting to the commerce platform or multiple commerce platforms, right? So we deal with a lot of companies that might have retailer, retail stores, and they have an online store, right? But you want to capture all of those conversion sources into that warehouse as well. So those three verticals, right? All of your advertising data, all of your customer touch point data, conversion source of truth, that’s flowing into a unified data warehouse. Then I can start doing some modeling, right? Then I can start unifying the data, put it in uniform data sets, and really start to model and build multi-touch attribution. We can start to build MMMs, media mix models, right? I can start to do incrementality testing. You know, I can build all kinds of customer insights from there.

I can build predictive LTV. I can show individual customer journeys, right? How the what their behavior looks like. And you get a much more complete picture. you of your marketing portfolio. You know, retailers if they do a lot of stuff with merchandising. you know, we can study how, you know, what actual SKUs and merchandise is interacting with marketing from an efficiency perspective. It’s a huge unlock. Right? Are you putting the right piece of piece of product in front of somebody? Right. And is it is it product they’re responding to and does it generate a really efficient marketing? you journey. So, you know, you it unlocks a whole other degree of visibility in intelligence layers that you just don’t have today.

Zach Thompson:
So I want you to expand on those a little bit for us if you can. Break those down a little bit further. So the multi-attribution modeling and the media mix. So

Ryan McClurkin:
Yep.

Zach Thompson:
can you talk a little bit more about those?

Ryan McClurkin:
Absolutely. So the way to think about multi-touch attribution, it’s you have a conversion, and then you have to ask yourself, what were the marketing touch points that led to that conversion? That’s the best way to think about it. A conversion happens. Whether it’s I generated a lead, whether I generated an order, you define the conversion, right? And I have to have visibility to it, the conversion occurring. And at the most granular level for that conversion for that customer, what marketing touch points led to it?

And then you have to have a data model in place that’s looking back and it’s looking at all those customer touch points and it’s saying, he got a piece of mail 20 days ago, he got, you know, a Google Touch 10 days ago, he saw some CTV impressions five days ago, and then he converted. Right? So can you piece together all of the marketing touch points that led to that conversion? And until you have that level of visibility, you can’t do multi-touch attribution there. way. Because you’re gonna end up having thousands of those conversions. Right. And then what happens is at the conversion grain, I can start to allocate that benefit across the marketing touch points that it created. That’s the multi-touch aspect of it. So what it is, it’s not an incrementality model.

It is directly allocating the benefit to the campaigns and touch points that led to that conversion. And it starts to explain the touch points that are happening that are resulting in conversions. Okay, so you start to get a feel for Here’s the campaigns, here’s the marketing touch points that are leading to conversions, and here’s the average efficiency coming from those channels. Okay, you and certain paths are more efficient than others, and you start to learn that. That’s multi-touch attribution. Most people do it very wrong. Most people their look back windows are far too short. Like I mean, I could show you client upon client where you know the average time to conversion is 30 some odd days. Okay. And it has standard deviations of seventy days.

Think about how flat that distribution is, right? Yet Meta’s over here trying to inform users on a seven day look back. you That’s a fraction of the benefit.

Zach Thompson:
Just reduce that as well.

Ryan McClurkin:
Right. No, they’ve they have a couple options, but I mean there’s like a 24 hour view through. Like that’s so small in the scheme of things and you have so many companies that it’s a considered purchase. Think about many but you if you’re selling a good that’s $500, you think they buy it in the twenty-four hours a year? No. No. Right? And so you start to see these longer journeys and you really start to get a feel for how long it actually takes the consumers. How many, how many times do they have to come back to the website and actually visit before they purchase? You talk about those brand impressions. It’s w it’s way more than what marketers actually think it is. Okay, and there’s been a lot of estimation. Until you have a really robust MTA, then you go, wow.

It’s taking my I’ve got consumers taking a hundred and twenty days. Hundred days. Right? And until they have that visibility, they don’t know how to speak to the org about it. And then when they can speak to the org, think about think about you think about how it changes the narrative inside the org. Right? Now when I spend that hundred thousand this week, I know, I have data that shows The average conversion is gonna come 30 days from now. So if you start cutting my ad spend, expect the impact on average 30 days from now, but it’s very spread out. I have this really long-tail distribution with these tails on it that are really big. Right? you I’ll feel some of it now, but if the average standard deviation is 70 days, it’s gonna keep going. Right?

you Or when I spin up a tactic, you’ve got to give it time. You’ve I got to have the right cost allocation models in place. So I don’t I don’t cut it off before it even has time to generate the benefit. Right? Then and that’s where it changes the conversation, the narrative. but that’s multi-touch attribution, right? You’re you’re sitting there, you generate a conversion, and you ask myself, what were the marketing touch points and how much did it cost me to generate that? We call that attributed spend. So everybody thinks about attribution from a benefit side and then they attribute the benefit to the touch points. There’s also you a concept of actually saying, hey, I know that touch point cost me $5, I know that one cost me $10, I know that one cost me $10.

That order or that conversion cost me $25. Right? Because I might have spent the money two months ago, but I’m gonna allocate it into the when the conversion happened. So we’re gonna attribute that spend to the conversions that are happening. Kind of a the financial concept sort of cost matching or you know, activity-based costing, very similar, where I’m gonna actually allocate the cost to the you benefit it creates. And then you don’t end up with that bullet. So that’s MTA.

Zach Thompson:
Before you move on, ask you quickly about that. obviously moving away from this kind of last-click attribution modeling that everybody has been using for so long, do you, with your product, are you guys giving people the opportunity or the option of establishing their own attribution, like decay-based attribution?

Ryan McClurkin:
Yes. After a period of time. Okay. Okay, so we have a myriad of I’m gonna say models available. Nobody in our system uses last click. Yeah. Right? They come to us to get out of that world. Right. Because once you actually see a multi-touch, you’re like, how can I ever how can I say that email data point or that brand search gets all the value when it took 10 other in marketing impressions to even generate that last one? Right. So I need to I need to place value elsewhere. So we typically start like say you’re new to us and you come in and we’ll start with like a linear, a uniform distribution. We’ll we’ll uniformly distribute that benefit.

But then what happens is over time, I can’t do it immediately, but over time when I have tens of thousands or hundreds of thousands of these journeys, I can start to put in a machine learning model that starts to weight the touch points appropriately. So eventually you can get to a place where I have sort of custom to my business, you right? How should I be weighting that benefit appropriately? Right. And some businesses are very different than others. Some of you never even get to the later touch points without generating the first one. So you better put a lot of value on the first ones. Right. And others more, you know, we see like for example, lower price points. We have a makeup brand for example. Low price point. It’s got a fast conversion window.

Last touch is not it’s you don’t want to be in last touch, but a lot of value goes toward the end touches. It’s it’s it’s it’s a faster but it’s not a high consideration purchase. You know where you have more considered purchases that are a little higher price points, we see that benefit get spread out more. But yeah you can absolutely get to that point. Can’t do it immediately, right? You need you need a lot of data to train the machine learning models to on those journeys to really start to allocate the. benefit the right way.

Zach Thompson:
You were going to jump next to I think through the three.

Ryan McClurkin:
Yeah, the mar marketing mix media mix model or marketing mix model, they’re the same thing. you know, the I always say the way to think about an MTA is very granular. You should be building it at the conversion grain, right? And touch points from there. Marketing mix models and media mix models, which are synonyms, they are built at a channel grain, and almost on a weekly basis. It’s very rare that you actually see one built at a daily basis. successes. So they are a much higher level model. And what they typically are, the industry standard that you see today, they’re they’re called Bayesian statistical models. This is area of statistics. But Bayesian MMMs, they are trying to estimate the incremental impact of that channel.

They’re run over long periods of time, usually two years of data, and weekly spend impression data, the benefit data, and what they’re trying to find are these correlations between spend fluctuations and benefit impact. And what you learn, what you start to learn from those is the estimated incremental return on spend that you’re generating from a channel basis. So what may happen is it depends on the business, but they may say, okay, I want to put all my I want I want my social upper funnel spends as a channel, or my social lower funnel, more retargeting, right? Or maybe I have an acquisition, a search acquisition channel, you right? And that might be a combination of Google non-brand search and Bing non-brand search.

And that flows into like this upper funnel kind of Google acquisition channel, right? And so what happens is the MMMs are estimating at that level. And what they help with is what I call portfolio level adjustments. So they start to give you some guidance on, hey, we’re seeing some really nice trends, for example, with search upper funnel. You should go spend more of your money there. So if I’ve got a twenty million dollar portfolio, maybe I need to make some portfolio level adjustments. I need to go spend seven more million in Google upper funnel because it’s got a really nice cost of acquisition, incremental cost of acquisition, and it’s generating a really nice incremental ROI. man. Right? And so you start to get into portfolio level adjustments, like a portfolio manager picking stocks.

But then when you go to spend that five million extra in or seven million extra in Google. acquisition, where do you actually spend it? Right? Because the MTA, the MMM is not gonna go to that grain. It’s not at a campaign grain. Never be built at a campaign grain. Right? I shouldn’t say never, there’s people that try to. But that’s where I go to MTA. So the MTA is measured at a very granular grain. You may have 25 different search campaigns that you’re participating in. And the MTA starts to give you indications of which ones are the most efficient. So if I’m gonna go actually spend more money or change my spend levels, then I jump into say an MTA world and go, okay, if I’ve got to s if I’ve got to allocate over the next year five more million dollars in this channel, where should I do it?

What do I see the trends in terms of the most efficient? And then I start to allocate spend. It’s also where incrementality testing comes into play. So that’s kind of the third leg of the stool. I always say there’s three legs to the stool. MTA is actually a core model because MMM should be anchored by something as well. So one of the things that you need to feed a an MMM or a media mix model is what you think the channel is doing. They call it priors in the statistical world. But when you don’t want to just blindly give an MMM and say, go estimate the channel ROAS. Right? Because it’s it’s running a bunch of simulations. That’s what it’s doing to try to get the impact of the channel. You want to anchor it.

You want to say, out of my MTA, I see the average return on this channel being $3.50. Start there. That’s your starting point. That’s your prior. And then estimate from there. See what happens. And it will try to find converge on an estimated ROI. Okay. So you always want to feed an MMM some level of estimated impact that you think is happening. So that’s the that’s you kinda I always say you almost start with like some people say, I don’t need an MTA. I just need MMM. You’re gonna struggle. Because what do you actually what do you give as a starting point for your MMM? Right? And then when you have to go spend the money from an MM perspective, how are you gonna determine where you’re gonna spend it? So those are the two those are two core legs for me.

They have you almost need one and the other, depending on what you’re doing. And then Incrementality testing. So what you’ll see come out of MMMs as well is that it can’t get a great read on a channel. So let’s say your Facebook spend’s been fairly consistent for the last two years. You haven’t fluctuated it, you haven’t turned things off or turned things on. The MMM is not going to get a great read on it. Because it doesn’t see any up and down and then the response. Right? you It can’t measure the response. And that’s where you say, okay, now I’m gonna go test, do some geo-based incrementality testing. So the gold standard for me is what I call synthetic control. It’s geo-based synthetic control testing. but it’s usually done at the designated marketing grain.

So in the United States, there’s 200 some odd, they call them DMAs. But you could think about Knoxville as a DMA. But it can be little regional bubbles, if you will. And you’re trying to measure the stability of those bubbles. And so you want to test where you have high levels of stability. Right? So if I’m a retailer and for example New York City’s a big DM, DMA. it might be a very stable DMA for me. I have a pretty consistent level of revenue and orders and customers coming out of that DMA. That would be a great testing DM. Versus say Charleston’s a small market and it’s doing this all the time. Right. You want to test in high s highly stable DMAs so I can actually get a response out of it. I you can measure the response appropriately.

Because if I don’t choose my DMAs the right way, I might choose a small DMA because I think it’s good or I can match it to this market. But the problem is that it has a high level of instability and my response may be within the normal error bound and I don’t really know what happened. Right. And so, you know, that’s that’s where you say, okay, if I’ve got to go spend more and Google. or you know, search acquisition, I wanna go maybe allocate that based on what I see in MTA, and I also want to do some incrementality testing.

Zach Thompson:
Break incrementality testing down for us a little bit more. So I give us an example

Ryan McClurkin:
So, you know, like I said, typically what we’ll do is we create what we call stability scores for the DMAs. The designated marketing areas. Think about them as cities. So that’s the right way to think about them. But they’re a little more regional than that. It’s actually a TV. Nielsen was the originator behind DMAs because they sell TV in markets. They’re just markets. And What we’ll do is we’ll break down over trending periods of time, usually a two-year period of time, we will break down how stable that DMA is from a orders, revenue, and new customer perspective. And what happens is we will then surface a selection of DMAs to the client, or and if you’re an operator in this space, you want to know which DMA should I go test in, right?

Because I may not want to go test in the Roanoke Virginia DMA because it’s too small and it’s highly variable. But My home market of Los Angeles, California, is very stable and it’s ripe for testing. you Right? And so what happens is, is we start to advise them, and this is how you should think about it as an operator, or as you go into the workplace or you are an agency, right? The DMAs you choose to do your testing in are incredibly important. Because if you choose highly unstable DMAs, you’re gonna be in trouble. You’ll never get a read. So what happens is you say, okay, I’m gonna set this, I’m gonna go double my Facebook spend in these markets. And you know, in the in the meta advertising world, you can choose by DMA.

So it’s a standard, it’s a very standard thing you can do is you can say, I want to turn on, I want a geofence, and just I want to test just the Los Angeles market, I want to test the Roanoke, Virginia market, right? And I’m gonna go double my ad spends in those areas. What you’re looking for is the response of the whole DMA. Right? You’re because you’re not doing attribution at this point, right? You wanna measure the orders revenue and new customers coming from that market. And do we see a spike? Or if I reduce that spend, do I see a drop? Right? And so what happens is if you choose highly unstable DMAs in the next, we usually run eight to twelve week tests, right? Because you need to give it time. Once again, you might have a response time associated with it.

I might double my spend, might not get the benefit the first week, then I start to see it rise over time, right? But what happens is if you choose highly unstable DMAs, you may get a bump or a decrease, but it’s within the normal fluctuation of that market. Whereas if I choose a highly stable DMA, I can look and say, okay, yeah, I’m I’m I’m now 10% plus from that market, and that market doesn’t have that kind of deviation in it. So we typically measure that’s how we typically set up the incrementality tests. but how you actually start to measure the benefit, it’s not just from what I call previous period results. W that’s where the I used a term earlier called synthetic control.

What we actually do is create they’re called statistical twins, but we’ll create from the two years of data we have, we’ll create a statistical twin to Los Angeles that you is supposed to behave just like it’s it’s it’s a model. Think about it almost like a regression model, and it’s and it’s predicting what Los Angeles would have done. Had you just done nothing? Just status quo. you Right? And then we’re gonna compare the actual result to the statistical tool. So it’s a little different than match market testing. A lot, a lot of people they do they do MMTs, they’re called match market testing, and they’ll they’ll pin like Los Angeles against New York. But like if you have a seasonal business or a swimwear brand, those might behave very differently, right?

’cause it’s gonna get colder in New York faster than it does in LA and then you’re trying to you’re trying to match markets that may not behave the same. So what we actually do is create a statistical twin that’s called the synthetic control of a market and we’re gonna measure the performance against that statistical twin.

Zach Thompson:
Wow, that’s absolutely fascinating. So we kind of talked on the problem that you’ve seen in the client side, brand side for years, the solution that you guys see now and providing them for those brands. Let’s talk a little bit about the future. So where do you see marketing and particularly analytics? Where do you see that going? What’s the role of AI now?

Ryan McClurkin:
Yeah, so the role of AI is really important. Okay. you know the data capture elements and such I don’t I don’t think are gonna change materially with AI. That’s not you’ve still gotta put in place the foundation for being able to do these measurement techniques. Y you know, a you can’t build an MMM model unless you have the data. You can’t build an MTA model, a multi-touch attribution model unless you have the data. Right. You know, can you even piece together incrementality testing without having visibility and set it up the right way? It still requires human touch. Because you’ve actually got to go geofence in Facebook and you know track it and do those things, right? So that’s not necessarily where I see the AI play.

Where I see the AI play is actually the consumption of the data and the results. One thing that has really come to light in the last four to six months are something called MCP servers. And you see them, if you use AI at all, you’ll hear about them. They’re called model context protocol servers. Okay, that’s what that’s what MCP stands for. And what it is, is that you’re giving the agent the ability to view a certain data environment with business context. That’s the context part. So, for example, we have quite a few clients, and what they do is we enable them, we enable what we call the Insighta MCP, okay? And what it’s doing is we’re enabling their team to connect to our data layer but not have to go through some fixed dashboards. That’s the way to think about it.

So we let them study those three intelligence layers using an agent. you And the agent, in our MCP server, we’ve written all the context for the agent. And you can think about them as skills. Some people think about agent skills. They’re not quite skills, they’re actually semantic layers is what we call them. But the agent goes in and reads the semantic layer about that data set and that data environment, that intelligence layer. And then they’re able then the user is able to then just through natural query language, just like you interact with ChatGPT or Claude or whatever you’re using, right, you can study those data that. And that is a massive enabler because I have just taken you away from what I call fixed dashboards, which you is which is how it’s been done for 20 years.

And now not only can you build a fixed dashboard at a single query or you know, you a single prompt, you can have it build you a fixed dashboard. But if you’re curious about certain areas. Like, I really want to understand the merchandise. I wanna find diamonds in the rough for my mer I sell a thousand SKUs. Find me some diamonds in the rough that are, you know, popping out that have really high return on ad spend, low cost of acquisition. Go find me the top five SKUs I should focus on. It can do that in a heartbeat. And do it in a very customized fashion. you Right. And so that’s where we see marketers really gravitating toward and operators. It’s not these self-service dashboards where you’re dragging and dropping metrics.

It’s now the ability to build fully custom dashboards you from a prompt. And you’re really enabling them to do that. And you think about how powerful a marketing warehouse can be in that manner, I’m capturing all my ad spend, all my customer touch points, and all my conversion data. In one environment. you That environment has to be AI readable or AI ready. That’s where people are not today. you Companies have data warehouses they built 10, 15 years ago, they are not AI ready. They can’t put MCP servers on it yet. Right? And I think that’s the transition you’re gonna see in the next five years. The next five years, you’re gonna see companies chasing making their data warehouse AI ready.

you Because when you enable and you unleash an org on top of that data warehouse and agents can read it, they are far better at processing that data than the human brain. That’s the reality. You still have to be guided by the human brain, but they can just parse it faster. You don’t have to have a human writing, they’re they’re far better at writing the SQL than we But you have to ha you have to write the contextual layers the right way so they don’t hallucinate. You know, that’s the biggest thing is like I think that’s where you’re gonna see a transition in software engineering and you know, database engineering is you’re not necessarily maintaining data environments like we used to or writing code in SQL.

It’s really almost like maintenance agents and writing semantic layers so that the agents can read the data properly. You wanna make sure they’re not hallucinating on you. But you know, we w you know, our warehouse is already AI ready. it’s just the timing of when we started and when we’re building, it just happens to be that way. But a lot of our clients and what we talk to and what we see in the space, the companies are not there yet. And they’re not gonna be there for another three to five years. And that would probably be on a fast timeline. It takes a long time to, you know, if you have an enterprise data warehouse to make that transition, but that’s where the companies are going.

Zach Thompson:
So what do they need to do over the next three to five years to make their data AI ready?

Ryan McClurkin:
That’s a that’s a large engineering task. I mean I th I think that’s where you’re gonna see data engineers spending time. Right? It’s not about building self-service fixed dashboards anymore. It’s about trying to enable an operator to be able to prompt the data the right way. And when you have a lot of it in different places, that becomes difficult. You gotta have really good data models. And so I think you’ll see that transition occur and then you’ll see companies I’m gonna say releasing their own sort of MCPs. Right. And a little bit of what we’re doing for those companies is building them a data warehouse that is AI ready and then giving them their own MCP. Right? They can go study all their own data.

And you know, the clients that are, you know, the companies that are really latching onto it, it’s an incredible Unlock. because now you have everyday people that aren’t sophisticated. They’re not technical individuals, but they’re really good marketers or they’re really good operators, they’re really good merchandisers. you But they can’t write SQL. But now I can go study that data. And I don’t even have to go, I don’t even have to like request it from a team and wait two weeks. That’s that’s the change. They’re getting the answer in five or ten minutes, right? And from a very sophisticated manager. And there’s a lot of customization that they can do in how they view the data. That’s the biggest, it’s one of the biggest unlocks we’ve seen in our space.

It’s especially in the last six months. It’s it’s moving so fast.

Zach Thompson:
You talked earlier about having the human in the loop. You still need someone at moment to help make those decisions. How far are we from the AI not just making a recommendation, but implementing it, allocating the budget for it?

Ryan McClurkin:
I mean I think I think you’re you already see you already see I’m gonna call them AI bidding agents and that kind of thing. It’s already happening, but I wouldn’t say they’re great yet, right? I th I think you know that’s all that’s already happening. It’s been happening for a while. I still think there’s this highly important someone still has to make a decision, right? And be accountable for that decision. or you don’t operate it as a business well, right? And the question is can you can I inform that individual in a far more informed manner than they’ve ever had before? And be able to get them data faster, be able to them to analyze data faster and conversion rates and ROAS and all their marketing data in a just much more faster fashion.

But you know, as far as like the pure I mean, I mean, you know, from a literal standpoint of The agent setting budgets. I mean, they’re already making recommendations, right? But somebody still’s got to pull the trigger. Somebody still has to say, okay, we’re gonna go spend the extra two million. And you still have to have an accountability for that, because you can’t fire Claude. Right? I mean, it’s accountability at the end of the day. Like who takes responsibility for that decision? Right? And because boards, CEOs, CFO, someone’s accountable, right? For the business performance.

Zach Thompson:
If you are a 22 year old now, you’re about to graduate college with a marketing degree or even a math or economics degree.

Ryan McClurkin:
Like an engineering degree even. Yeah. What would you do?

Zach Thompson:
to prepare yourself knowing what you know now. What would you be doing to prepare for a career in this space?

Ryan McClurkin:
I think you really have to be diving into the well, okay, so I think no matter whether you’re technical or more on the marketing side, you really need to figure out how to leverage the agents in your world. The AI agents. They don’t necessarily I’m not gonna say they’re they just do the work for I think there’s this they have a little bit of a stigma. I think especially in the EDU space, right? well I write my paper with AI. No, that’s not what I’m talking about here, right? What I’m talking about is can you leverage those agents to help you do your work and 10X you as a value individual? Like we call five or ten Xers. Like we run into engineers that are five or ten Xers that are using agents.

And the reason is because when I’ve got to go write SQL or a data model, they are better at it than a human. in terms of actually writing the code. I don’t need a human to spend their time writing the code. They’re far better at it. But they have to be guided by a human. Because there’s still this business context that you can put the pieces together. You have to almost think about it as an architect. Right? you’re the architect piecing together the elements and you still gotta be able to read the code and QA the code. you But in terms of actually writing each individual line, the agents are far better at it. Far, far better. And they can test faster than you’ll ever test. And that’s one of the things.

Like so coming from a technical background, analytics, marketing analytics, data engineering, you have got to figure out how to leverage those agents in a And I’m very meaningful way. Because that’s how you five or 10X your value to the org. Right. And it’s it’s so funny when I hear about the big orgs, the enterprises like. We don’t allow we don’t allow AI in our business. You are gonna get left behind. Someone’s gonna come nipping at your heels and they’re gonna blow you away. Because the agents are so good at the coding aspect. I mean that’s what they were built for. Not writing essays. They were built for coding and math. That’s what that’s where they live, right? And so they’re so good at that, but you still gotta be able to piece it together.

You still gotta have that architectural brain to be able to say, okay, I need it to do this thing so it fits into my bigger model here, right? And that’s an architectural role. And so but you’ve gotta figure out if you’re a junior engineer or in analytics, how can you leverage those agents to make you ten times more efficient? Because you can’t. And that’s what I see the smart ones doing. I love it when they’re when they’re I mean I If I can hire somebody that’s a five or ten X in the output of me hiring somebody else, I’m taking them all day. All day. Right? I think on the operator side, so let’s go less technical. Now let’s go to marketers or junior marketers, right? Analysts.

you Lever learning how to leverage the agents, build skills, build dashboards, dive through data, incredibly important. you Because They’re better at a human and faster than a human in terms of dissecting all the data. You think about like it’s like an onion, right? And gotta peel every layer off. They’re far better at that, but you’ve got to be able to guide it. And you should you should have a deep understanding of how to use something like a cloud or a or an OpenAI to its full capability. You can you can 10x your value to the org by Bye. being able to leverage these agents to study data and study different sources and connect the pieces. Right.

I mean we have we have clients building entire email nurture flows you using Claude versus having to wait two weeks for creative team and this and that and know what to piece together and then they’re you’re using, you know, the Klaviyo MCP to just push it straight to Klaviyo. And you and what they’ve done is they’ve taken their email team has now built fully personalized emails and they can handle the workload. It’s not that they don’t fire anybody. It’s just what they did is they train their teams on how to actually leverage the agents to create meaningful value. And I think if you’re a young person in this space, you know, there’s such a temptation, I think, for them to use it for schoolwork. No, no, no, no. Learn it. Like learn how to create skills. Learn what skills do.

Learn how to go Connect to an MCP and what an MCP is and go connect some of those. Go pull data down. Figure out how to dive deep through it. Prompt it the right way. They call them prompt engineers. Like there’s literal, there’s like literal prompt engineers. Like figure out how to leverage that the right way, and you will just 10x your value when you come to the workplace. because the ones that aren’t, like they’re gonna get left behind. They’re they’re they’re they’re they’re gonna get stuck in the and it and it’s moving really fast. So what will happen is it’ll happen really quick. you’ll know who’s doing it and who’s not and leveraging it the right way. And you know, I think a lot of people like, I can tell it’s written by AI.

I mean, I’m I’m just telling you, it’s it there are ways to train the agents to have a vocabulary, to have a voice, right? But go learn those things so that you can bring a ton of value to the org. you and I you know, I th I think as students they think about it very simplistically in terms of I’m just trying to get through this assignment. Like let me use it for that. No, no Like learn what it’s doing and how you can leverage it in the workplace. Cause you’ll bring ten times more value.

Zach Thompson:
Alright, so learn to leverage the agents, AI proficiency essentially for.

Ryan McClurkin:
Absolutely. Yeah.

Zach Thompson:
So what in terms of skills, right? But then what knowledge do you have now that you wish you had at 23 years old?

Ryan McClurkin:
man, it’s ch it’s just changed so much, right? I think I think you know, in the marketing space, I think understanding the problem you’re gonna run into as well with like just measurement. So agents will solve the measurement problem, right? But when you’re a marketer and you’re in the workplace or you’re at an agency or you’re in the workplace working on a marketing team. and you’re responsible for a certain piece of budget or you know you’ve got to go spend the money or start helping make those budgeting decisions. Like is your measurement stacked the right way? I you know, I went through a learning phase, right? And I think every marketer and every analytics person will. But take away from the beginning of this conversation, like, just think about it, am I making those mistakes?

Those are common mistakes. And they’re really hard to overcome. Like am I you just using a click based model? Am I using a GA4 output? There’s a better way. Right? Am I capturing the data in an automated fashion or am I having to go pull a Facebook report and put it in Excel? There’s a better way. Right? And so thinking about, do I have a media mix model? I got a multi-million dollar budget. Do I have a media mix model? Or am I kind of guessing, getting a little gut feel, right? There’s a better way. Right? Are we doing incrementality testing? Are we doing geo holdouts? Are we doing audience holdouts? Right? Can the advertiser do audience holdouts?

So I think having You know, that bigger picture in mind as you enter the workplace and you go to a workplace and go, okay, you know, we’ve got some measurement in place. Let me go evaluate that. Do I have any view of the lifetime value I’m creating for the oracle? Right? What value is being created by this marketing? Is it just all immediate or am I actually creating future benefit? Right. And I and I think is, You know, I wish I could hear a talk like this when I was 22 entering, because man, I it would have been like, man, we can accelerate this process, right? you know, that I think I think being open and having going and pushing that change is really important because you like I said, almost every org I talk to is struggling this story. in that space.

And it’s and they’ve had it for 30 or 40 years. And that’s one of the hard parts you’re gonna overcome. A lot of the CMOs have been ingrained. You know, now your CMOs are calling 40 to 60 years old, a lot of 40 to 60 year old CMOs. They grew up in a digital era that was click-based last touch. So how do you transition an org from that to a true multi-touch advanced measurement system. Right? That’s a tough transition. But you’ve you got to start thinking about it, having those conversations and bringing it up. And it like I said, it’s a learning phase. It’ll it’ll happen.

Zach Thompson:
over time. Fantastic. Where can people find out more about you and the work that you’re doing at Insighta?

Ryan McClurkin:
You know, we got website Insighta.io. It’s like insight with an A dot IO and I’m on LinkedIn as well. But yeah, happy to ever talk to anybody that’s in this space and teach them more about the space. I d I’d like I said I talk at a lot of conferences, I give keynotes, You know, I’m interested, I always say when the brand wins, everybody wins. So, you know, I don’t even necessarily I’m not always interested in somebody buying my problem. I we might not be the right fit, but I’m but I’m always interested in helping people. So I’ve helped a lot of brands and just point in the right direction even. Or students and people going into agency world. A lot of agencies, you know, they need they need an analytical backbone or a different way to think about it, right?

And so I’m happy to have those conversations and help out.

Zach Thompson:
That’s great. Thank you so much for joining us. Absolutely. are going to get so much out of this. Really appreciate your time.

Ryan McClurkin:
Absolutely. Thank you.

Zach Thompson:
Folks, if you want to hear more on conversations like this, like and subscribe with The Marketer Exchange for further conversations.