Econometrics or Marketing Mix Modeling (MMM) is a powerful tool for deciding where to spend your marketing budget. It uses Linear Regression to associate spikes and dips in sales back to actual events, for example price increases, changes in the weather and spend on advertising.
This is an alternative to attribution modeling, which measures at the user level to assign credit for each purchase. For example in Google Analytics you can see how many conversions came from Facebook ads on a ‘last click’ basis — that’s attribution modeling. The Econometrics approach would look at increases or decreases in conversions relative to how much you spent on Facebook, taking into account the ‘halo effect’ not captured with a last click model.
So how do you choose whether attribution modeling vs econometrics? No one approach is applicable to every situation, and smart marketers often use them both for different reasons. Econometrics tends to be more useful on the strategic level, but often doesn’t provide much granular insight. Whereas attribution modeling is extremely granular making it useful for optimization, but alone it can lead to over-investment (or under-investment) in certain channels.
What questions can Econometrics answer?
There are several recurring questions that every marketer eventually faces, where Econometrics has a good chance of getting you an answer.
Why doesn’t Facebook match Google Analytics?
This is a classic example of a question that attribution modeling really struggles to answer. Any marketer running ads will quickly realize that what the ad platform is reporting doesn’t match what their own analytics is telling them. Using our example of Facebook ads and Google Analytics, there are several reasons why attribution fails.
- View Through: Facebook counts conversions that came 24 hours after someone viewed an ad — Google Analytics doesn’t have access to this view through data
- Click Window: Facebook counts all conversions that came within 28 days of a click on an ad (even if they came back to the website via another channel later) — Google Analytics only gives credit to the last click before a sale
- Date Assignment: Facebook assigns a conversion back to the day the ad ran — Google Analytics registers the conversion on the day the conversion happened
As you can see there are a number of reasons why attribution might not be perfect. Ultimately you’re left trying to decide who to trust, Facebook or Google? Both models are biased in some way based on the incentives of the companies who own them. Facebook probably didn’t drive all of those ‘view through’ conversions, but we also can’t argue view through has zero value.
Econometrics stands apart from platform bias — you run the analysis on your own data, so you don’t need to rely on Facebook and Google to tell you the truth. It will give a fair assessment of what the incremental impact of Facebook ads actually is. Usually this takes the form of a ‘multiplier’, for example it’ll tell you that 40% of Facebook’s view-through conversions were incremental. So you can make strategic budgeting decisions by down-weighting Facebook’s ROI by 0.4, and invest accordingly.
Note this isn’t just useful for when Facebook ads and Google disagree — any time a channel might be under or over weighted based on the way it measures performance, Econometrics can help. Examples include retargeting ads (which are targeting people likely to convert anyway), brand keyword campaigns on Google ads (which attract people already familiar with you) and podcast advertising (where users often come months later having forgotten the discount code).
If I doubled my marketing budget, what would be my ROI?
This is a common question, especially in ambitious, fast-growing startups. They want to know if they increased their investment, what would they get for it? You want to check that your marketing goals are realistic, and if you’re wise to the impact of diminishing returns, you’ll want to make sure you aren’t setting yourself up to fail. This can be useful to make the case for more budget, but also to determine which channels should get that budget — what is the optimum marketing mix for your business to maximize ROI?
This is something attribution can’t tell you — it’s focused on telling you how well a channel is performing right now, but can’t easily extrapolate out to the future. Thankfully Econometrics can tell you not just how much your cost per acquisition will increase per $1 of ad spend, but also let you forecast what your cost per acquisition will be at different budget levels to make your decision. As a caveat, I should mention it’s important not to extrapolate out too far without testing at higher spend levels — the further out you go the more accuracy suffers.
How do I measure the impact of TV?
For many data-driven startups, they find the level of control and optimization they’re used to with digital ads just doesn’t exist on TV. There’s no way for the user to ‘click’ to their website, so in effect everything is ‘view through’! In addition, TV tends to work more as a branding channel rather than direct response — it might take months for a sale to happen after viewing a TV ad, rather than minutes, hours or days.
Econometrics can tease out these complex non-linear effects, and show you not just how much TV drives your sales, but also over what timeline and payback period. It can even show you how a campaign for one product might impact sales of another product — an effect that most marketers are positively surprised by. Of course as well as TV, Econometrics is a good solution (often the only solution) for measuring other offline channels, like billboards, radio or press.
What was the impact of COVID-19 on my business?
As marketers, we’d love to think we’re in control of everything that impacts our businesses’ growth. Unfortunately that just isn’t the case — as all of us were reminded by the impact of COVID-19. For some businesses it was as much as a 90% drop in performance, for others particularly in ecommerce, they saw skyrocketing growth. What is harder to tell is precisely how much positive or negative impact COVID is having — was it really your agency’s smart maneuvers that led to increased sales or were they just riding a macro-economic wave?
Of course COVID-19 isn’t the only external economic impact your business faces. The growth rate of your industry, demographic trends, even weather can make an impact on your sales. Most Econometricians attempt to work these factors in. Another important factor to consider is competitive behavior — is your biggest rival suddenly spending on TV? It’s often possible to measure or estimate competitive activity and work this into your model to explain dips or spikes that otherwise would have been unexplained in your marketing attribution model.
How do I measure performance when users don't want to be tracked?
There is an ongoing privacy backlash against the surveillance capitalism that powers the online advertising industry. Safari and Firefox have removed 3rd-party cookies used for marketing attribution, and even Google Chrome is following suit. Around a third of website visitors use Adblock and Apple is killing the IDFA tracking id that enables marketing attribution on mobile apps. On top of that the GDPR legislation in Europe and CCPA in California put you at risk of a fine if you violate users privacy, even for the purposes of marketing attribution.
Thankfully Econometrics can help step into these gaps and give you some way to make strategic budgeting decisions without the user-level data you're missing. Econometrics is fundamentally a privacy-friendly attribution method — one that will always work even if 100% of your users were anonymous. As the era of big data comes to an end with users increasingly opting out of tracking, companies will need to rely more on Econometrics and Marketing Mix Modeling to determine the effectiveness of their campaigns.
When does Econometrics not work?
Of course Econometrics can’t solve every strategic marketing question for your company. Often you’ll find that the actions you want to take can’t be predicted based on the data you have, and you’re left with attribution modeling or even ‘gut feel’ to guide you.
User-level data
This is the biggest downside of Econometrics — because it only uses aggregate data, it can’t tell you about any individual user. With attribution modeling you can pull up a list of users that came from a specific channel and visually inspect the data, send campaigns to them, whatever you need to. With Econometrics you only see a high level view, so it can’t tell you anything about individual users.
Of course the usefulness of user-level data is a garbage in, garbage out situation — unless you’re really confident in your attribution model, your data might not be that actionable. There’s a difference between being precise and being accurate. However if you really do need user level data for whatever reason, then Econometrics isn’t the right tool for the job.
Optimization in real-time
Often building an econometric model takes anything from hours to weeks or months. It can be particularly slow for large, complex businesses that have many stakeholders and non-standard data sources. Often just cleaning the data can take 80% of the time allocated to the project, so most practitioners run a model only once or twice a year.
Additionally because Econometrics tends to look at things in macro, the big picture, it often finds no significant impact for smaller changes, making it better for strategic rather than tactical decisions. When you change your bids on Facebook ads or update an ad with a different call to action, you aren’t likely to move the needle enough to register in an Econometric model, but these activities can still be very important to driving ROI.
Brand New Activities
Econometrics relies on past data to look for patterns on what the future could look like. The past can be surprisingly predictive of the future, even with simple models. However if we’re trying to predict an activity that’s entirely new, we might find that Econometrics doesn’t have much to say about it. For example a new marketing channel might perform very differently from our existing ones, a new creative route might perform far better (or worse) than expected from past creative tests, or increasing ad spend by a considerable amount might lead to rapidly diminishing returns that we hadn’t seen in the data to date.
---
If you’re interested in getting started with Econometrics, or want to ask a few more questions, don’t hesitate to reach out and ask. @hammer_mt
If you’d like to learn more about Econometrics, check out the rest of the posts in the series:
- Econometrics in GSheets
- Econometrics in Python
- Diminishing Returns
- Measuring the Halo Effect
- When Econometrics Works (this post)
In future we plan to cover:
- Word of Mouth Coefficient
- Testing Model Accuracy
- Monte-Carlo Simulations
- Machine Learning Models
- Data Import from APIs
- The Future of Econometrics
- Any other topics? Tweet @hammer_mt to request
.jpg)