129 pieces · 13 outlets
Elsewhere
Most of what I’ve written lives on other people’s sites. This is all of it in one place, grouped by where it ran, newest first, with a line on what each piece is about. The books have their own page.
Every
Head of Evals · Since 2026
Head of Evals, after a stint leading AI tech consulting, and author of the Also True for Humans column.
Visit every.toThe Best AI Agent Builder Is Trapped Inside Microsoft
Argues that Microsoft Copilot Studio is a strong no-code tool for building agents that call other agents in enterprise workflows, held back by Microsoft's confusing Copilot branding and user experience.
Every
AI Could Do Anything. Then It Met PowerPoint.
How Every built an AI workflow for client-ready PowerPoint decks after Claude and Codex skills fell short, using a blueprint-first skill in its Claudie assistant, then adapting it for a client making 25 decks a week.
Every
I Interviewed an AI Version of GitHub’s COO—Then Spoke to the Real One
Before interviewing GitHub COO Kyle Daigle, the author built an AI persona of him from public material and asked it the same 12 questions; two answers matched, four partly matched, and the misses showed where to dig.
Every
How Microsoft Is Building for a World of Metered Intelligence
A report from Microsoft Build on how Microsoft is responding to rising AI costs after moving GitHub Copilot to token billing, covering automatic model routing, smaller in-house models and hill climbing with evals.
Every
The Eight Levels of AI Adoption
A guide co-written with Laura Entis that maps eight levels of AI adoption, from chatbot and copilot through multi-agent work and orchestration, with sample prompts for each level and signs of when to move up.
Every
You Are the Most Expensive Model
Introduces incremental determinism, a four-level process of turning repeated AI sessions into skills, skills into evals, and evals into scripts, so costly models and human attention go only where needed.
Every
Your Best AI Strategy Starts at the Top
Argues AI adoption has to start with executives using the tools themselves, since AI works more like a new employee than software, and shows how to codify what top staff know as reusable skills.
Every
Vibe Check: Cursor 3.0 Bets Big on Agent Orchestration
A team review of Cursor 3.0, a rebuild around agent orchestration, finding a fast app and a capable Composer 2 model but missing basics like a file explorer and branching that keep it behind Claude Code and Codex.
Every
Seven Things I've Learned Getting Companies to Use AI
Seven lessons from consulting on getting teams to adopt AI, including buying models directly, workshops that are at least half build time, assigning impossible tasks, and training staff to manage agents.
Every
I Achieved the Four-hour Workweek. So Why Did I Just Take a Job?
Why the author gave up passive income from a prompt engineering course to join Every full time, arguing that AI makes building easy but finding customers hard, and that learning is faster inside a company.
Every
Why I Turned Off ChatGPT’s Memory
Explains why the author keeps ChatGPT memory off, using personal examples of skewed answers and a taxonomy of context failures (poisoning, distraction, confusion and clash) that make results hard to control.
Every
What AI Is Teaching Us About Management
Argues that working with AI agents trains people in management, calling the overlap New Taylorism, and shows how prompting, orchestrating agents and deciding what to build mirror delegation, project management and strategy.
Every
This Prompt Optimizer Learns From Its Mistakes Like DNA
Explains GEPA, a DSPy prompt optimizer that mutates and selects prompts like natural selection, with a 44 percent quality gain for Every's Spiral and an invoice sorting case where it doubled accuracy in 20 minutes.
Every
I’ve Stopped Writing Prompts—DSPy Does It Better
Why the author now uses DSPy to optimize prompts instead of writing them by hand: you define inputs and an evaluation metric and it searches for better instructions, introducing a technical lecture with code.
Every
The Most Human AI Model Is Also One of the Cheapest
Compares 12 models on roleplaying real people across five replicated market research studies, finding that accuracy and sounding human differ, with cheap Gemini 2.0 Flash the most humanlike and GPT-4.1 the most accurate.
Every
When AI Gets It Wrong—And How to Know
Three situations where AI persona feedback reliably misleads: outdated knowledge, favoring socially approved answers over real behavior, and lacking hands-on experience, with fixes for each drawn from Ask Rally data.
Every
I Cloned 2,000 Hacker News Users to Predict Viral Posts
Tests 1,903 AI personas cloned from Hacker News commenters on predicting which headlines go viral, finding 60 percent accuracy, and explains why social dynamics like early upvotes limit how much better prediction can get.
Every
AI Focus Groups—And Soon AI Copywriters—Will Make Ads Superhuman
Lays out five levels of autonomous AI advertising, from roleplaying customers to self-improving ad engines, drawing on building the Rally virtual focus group tool and evidence that AI writing now passes Turing tests.
Every
I Created a Hacker News Simulator to Reverse-engineer Virality
Builds a Hacker News simulator from personas inferred from real commenters to test headlines against trending stories, compares models on response diversity, and shows how to get blunt rather than polite feedback from AI.
Every
I Asked 100 AI Agents to Judge an Advertisement
Introduces personas of thought, a prompt that simulates many audience members to get less generic feedback, applied to Jaguar's Copy Nothing ad, to cloned customers in NotebookLM, and to a script that polls 100 agents.
Every
The No-fluff Guide to AI Agents
A plain explanation of how AI agents use tools, from function calling to the ReAct method of combining reasoning with actions, and where computer use and humanoid robots could take agents next.
Every
When Guessing Isn’t Good Enough
Explains retrieval augmented generation and vector search as a fix for hallucinations, covering embeddings, chunking strategy and a FAISS demo on a fictional employee handbook, and how retrieval could give AI assistants memory.
Every
AI Never Gets Tired of You Asking
Explains self-consistency sampling, running one prompt several times and taking the most common answer, starting from the author's habit of briefing three designers at once, with a math word problem example and when the cost pays off.
Every
The Key to Great AI Prompting? Show, Don’t Tell
Covers few-shot prompting, giving a model examples to imitate, with research on gains over zero-shot, a Steve Jobs style product naming demo, and the tradeoff between reliability and creativity when choosing examples.
Every
AI Works Better When You Make It Pretend
Examines role prompting, why studies show mixed results on accuracy, and argues its value is steering style on subjective tasks, shown by a test where a role plus examples raised Elon Musk style product names from 25 to 88 percent.
Every
Why AIs Need to Stop and Think Before They Answer
Explains chain of thought prompting and why letting a model plan or reason before answering improves accuracy, with research results and a Fermi estimation example, and argues cheaper reasoning will make agents more capable.
Every
Reverse-engineer Your Creativity With AI
Uses GPT-4V to label 564 coffee shop photos and measure which design cues are common or rare, as a method for deciding what to copy from competitors and where to differentiate.
Every
How to Become an Expert at Anything With AI
How beginners can shortcut expertise by having ChatGPT find patterns across many examples, shown by scraping top Every article titles to generate title templates and by summarizing a site's articles with Claude 3.
Every
How to Grade AI (And Why You Should)
Makes the case that cheap, task-specific evals matter more than public benchmarks, comparing programmatic, synthetic and human evals by strengths, weaknesses and examples, such as an emotion prompt that made posts 13 percent longer.
Every
Emotion Prompting: Why AI Responds Well to Threats
Why emotional appeals and threats in prompts can change model output, citing research with a 10.9 percent average gain, a test getting Golden Gate Claude to stop mentioning the bridge, and the ethics of being mean to AI.
Every
The Ultimate Guide to Prompt Engineering
Five principles for better prompts (give direction, specify format, provide examples, evaluate quality, divide labor), worked through on a ChatGPT blog post example, with the argument that prompting skills will outlast any single model.
Every
Ask Rally
Co-founder · 2025
Co-founder of Rally, an AI persona tool for synthetic market research: simulated audiences calibrated against real people.
Visit askrally.comSynthetic Research: The Secret Ingredient is People
Tells how Rally moved from purely generated personas to GenPop, a panel of AI personas trained on interviews with real consenting people, to fix AI-sounding answers, politically correct bias and overly uniform responses.
Ask Rally
How We're Making Enterprise Pricing Research Cost Less Than Your Domain Name
Makes the case for running Van Westendorp pricing studies on AI personas for about $50 rather than thousands. Optimized personas match exact human price brackets 43% of the time and reach the same decision as a real Vexpower study.
Ask Rally
Optimizing to Remove Political Bias from AI Models Removes Other Types of Bias Too
Tests whether a prompt optimized with DSPy's SIMBA only on election votes also changes other answers. It moderates lopsided views on climate, free college and energy, barely moves movie and car choices, and reverses on offshore drilling.
Ask Rally
What's Predictive in a Persona?
Tests which of 25 persona attributes, grouped as demographic, psychographic, descriptive, behavioral and contextual, best predict which interviewed prospects became Rally users. Behavioral plus jobs-to-be-done context performs best.
Ask Rally
My Co-Founder Proved Me Wrong
A short account of settling a founder disagreement over a homepage button label by testing both versions on 64 AI personas cloned from customers, and what it taught about status quo bias.
Ask Rally
The 'Dutch Chris' Problem
Uses the founders as a thought experiment to show how rebuilding personas from aggregate statistics can attach the wrong traits to the wrong person, and how LLM-inferred correlations plus calibration reduce the error.
Ask Rally
AI Broke Entry-Level Hiring
Argues that AI has removed the simple tasks that used to train graduates, and draws on the robotic surgery precedent and Ender's Game to propose realistic AI simulations as a new training ground for new hires.
Ask Rally
The Intention-Action Gap
Asks whether AI research should match what people say or what they do. In an eco car test, Claude Haiku personas pick the green option 78% of the time while Sonnet picks it 37%, closer to real buying behavior.
Ask Rally
How To Replicate Tim Ferriss's Famous Title Test Using AI Personas
Recreates Tim Ferriss's AdWords test that chose the title The 4-Hour Workweek, using 50 AI personas and disguised title equivalents, and focuses on what the personas' stated reasons reveal that click data never could.
Ask Rally
Van Westendorp Price Optimization
A playbook for running the Van Westendorp price sensitivity survey on AI personas, with the four questions, how to read the price intersection points, a starter prompt, and the founder's own Vexpower price cut story.
Ask Rally
Correcting Bias in LLMs with DSPy
Uncalibrated AI personas voted 90.6% for Harris in a 500-persona election test. Compares three DSPy optimizers and finds SIMBA brings the split to within about two points of the real distribution.
Ask Rally
Cheaters, Speeders, and Repeaters
Looks at three kinds of fraud that corrupt paid market research, cheaters, speeders and repeaters, how researchers catch each, and argues that small, clean human studies can calibrate AI personas used at scale.
Ask Rally
Personification: Combining Disconnected Datasets To Construct Realistic Synthetic Audiences
Explains personification, a method based on the SYNC Gaussian copula approach for merging census, customer and survey datasets into synthetic personas whose attributes add up in aggregate without implausible individual combinations.
Ask Rally
Changing Artificial Minds By Feeding Them Biased Media
An experiment that feeds 100 census-based AI voters pro-Trump or pro-Harris articles to see whether media diets flip a simulated 2024 election, and shows how strongly LLM personas lean left before calibration.
Ask Rally
Predicting Gamer Preference With 85% Accuracy
Replays four mobile game design tests from PickFu with 50 AI gamer personas in Rally. The personas pick the human winner in three of four tests and match vote shares with about 85% alignment.
Ask Rally
Personas of Thought: How to Use AI Crowds for Better Decisions
Introduces Personas of Thought, a prompting technique that asks an AI to role-play a set of distinct personas, answer as each one, then merge their views. Includes copy-paste templates for headline, product and audience testing.
Ask Rally
Guest posts and elsewhere
Newsletters, other people’s blogs, my old Medium and the odd book.
Five proven prompt engineering techniques (and a few more-advanced tactics)
Five prompting tactics with templates and product examples: role-playing, style unbundling, emotion prompting, few-shot learning and synthetic bootstrap, plus an introduction to more advanced multi-step techniques.
Lenny’s NewsletterPartly paywalled
How to evaluate AI performance when you have no domain expertise
Lessons from a client project on improving AI output without domain expertise: logging specific failures with the expert, explaining hallucinations and temperature, A/B testing prompt changes 10 to 30 times, and training an LLM judge.
LeadDev
Prompt Engineering from 2020 to 2025
A timeline of prompt engineering techniques from GPT-3 few-shot prompting in 2020 through chain of thought, ReAct, LLM-as-a-judge and automated prompt optimization, with foundational tips and predictions for 2025.
AI Supremacy
How close is AI to replacing product managers?
A blind test of AI against human product managers on three tasks (product strategy, metrics and ROI estimation), where voters preferred the prompted AI answer in two of three, with the prompts shared.
Lenny’s Newsletter
Putting a Prompt into Production
How to move a prompt from trial and error to a production system, applying the five prompting principles and a worksheet template to a forum spam detection case study, with cost tradeoffs between GPT-3.5 and GPT-4.
Reforge
thumb: A Simple Prompt Testing Library for LLMs
The README for thumb, a Python library for A/B testing LLM prompts, which runs each prompt 10 times, collects blind ratings and reports score, token use and cost.
GitHubOpen-source library
How today's top consumer brands measure marketing's impact
Co-written with Michael Kaminsky, a guide to the three ways brands measure marketing incrementality (multi-touch attribution, marketing mix modeling and conversion lift studies), based on research into 42 consumer brands, with steps to set up each.
Lenny’s NewsletterPartly paywalled
Marketing Memetics (book)
A self-published book on applying memetics to marketing, sold as a $9.99 PDF and EPUB, with companion essays published on marketingmemetics.com.
GumroadBook, on Gumroad
A Word of Mouth Coefficient Template to Solve Dark Social
A Google Sheets template that applies the Word of Mouth Coefficient to Rockerbox data, estimating dark social traffic by relating new direct users to active users, with steps for using your own data.
Rockerbox
Meme Mapping: Learn to Run Better Creative Tests by Reverse-Engineering Hollywood
Argues that creative is the last lever left as ad platforms automate targeting, and lays out Meme Mapping, a four-step method for cataloging and testing creative elements, drawn from patterns in Hollywood movie posters.
Reforge
Marketing Mix Modeling Template to Create an MMM Feed with Rockerbox Data
A contributor post with a Google Sheets template that turns Rockerbox data into a marketing mix modeling feed and runs a simple linear regression, explaining coefficients, p values and accuracy charts.
Rockerbox
AMA: I was the co-founder of a 50 person marketing agency...
An Indie Hackers AMA by the co-founder of a 50-person marketing agency who left to learn to code and whose edtech product had just had its first $1k week.
Indie HackersAMA thread
Fired for the last time
A personal essay on being laid off for the third time in seven jobs, deciding to stay self-employed, and launching Saxifrage as a solo growth studio for consulting and products.
Medium
10 of the best ads of all time, reimagined as Facebook ads
Ten famous ads, including Think Small, Just Do It and Think Different, mocked up as Facebook ads to test which ideas survive the format, concluding that the best ideas work anywhere.
Medium
14 Facebook ad examples from 7 years and $40m in ad spend
Fourteen Facebook ads from the author's career, drawn from about $40m in ad spend, each with notes on why it worked, from Money Dashboard and SumoMe to Booking.com and Monzo.
Medium
Marketing Plan And Chill?
A worked example of a full startup marketing plan for the movie recommendation chatbot And Chill, covering 15 sections from personas and SEO to referral triggers, PPC and budget forecasting.
Medium
Things for Brits to do in New York
A Brit in New York's guide for British visitors, with practical tips on getting around, plus recommended sights, neighborhoods, food and bars across the city.
Medium
5 Quick and Dirty Tricks for A/B Testing on Facebook Without Wasting a Penny
Five budget-saving ways to run Facebook ad tests, from judging on clicks instead of conversions to bandit, deprivation and geographic tests, built around a rule of thumb of 30 observations per variation.
Medium
Outsourcing: You're Doing It Wrong
A short argument that managers outsource the wrong work: hand off problems you already understand and can write a playbook for, and keep the fuzzy ones, like marketing, until you do.
Medium
Marketing Will Cease to Exist
A Quora answer arguing that the internet removed the information gap mass media marketing filled, so within ten years product quality, not traditional marketing, will be what moves the needle.
Medium
Things for Americans to do in London
An English expat's list of tips for American visitors to London, covering transport, sights, local markets, restaurants, bars and clubs, with specific recommendations and tourist traps to avoid.
Medium
Good, Cheap and Fast
Why bosses push for good, cheap and fast all at once, explained through the project management triangle and the principal-agent problem, and why being dependable and flagging risks early earns trust.
Medium
Ladder
Co-founder and COO · 2014–2020
Co-founded a growth marketing agency that grew to 50 people in three countries and ran thousands of experiments for clients.
Visit ladder.ioFrom Growth Hacker to Prompt Engineer
Maps the growth hacking process onto prompt engineering, arguing both rely on rapid experimentation, data-driven decisions and creativity, and that growth marketers are well placed to move into AI work.
Ladder
The Only Event Tracking Setup Guide You’ll Ever Need
A guide to writing an event tracking plan a developer can implement, covering event naming, tag management, funnel stages and mobile, with step-by-step setups for Unbounce, Ghost, email lists, WordPress and Shopify.
Ladder
Ladder’s Secret Weapon: Growth Engineering
What growth engineering is and how Ladder used it, through anonymized case studies from an experiment platform to e-commerce tracking, custom attribution and machine learning creative tagging, now handled with partner Saxifrage.
Ladder
Engineering Virality into our Pre-Launch Campaign Drove 42% More Leads (Play-by-Play)
A play-by-play of the pre-launch campaign for Ladder's Emerging Channels Report, where referral sharing added 383 of 906 total leads, a 42% lift, with templates, tech stack and costs per channel.
Ladder
Plan for Growth with the Monte-Carlo Simulation [No Code Solution]
An introduction to Monte Carlo simulation for marketing forecasts, replacing a single dragged-down spreadsheet number with a range of likely outcomes, built without code in the modeling tool Causal.
Ladder
How to Use Facebook’s Dynamic Ads For Quality Blog Traffic
A setup guide for promoting blog posts with Facebook Dynamic Product Ads, treating posts as a product feed and optimizing for readers who get halfway through an article, shown on a Webflow blog.
Ladder
An Expert’s Thoughts On How To Scale ‘Single Keyword Ad Groups‘ (SKAGs)
How to balance keyword stuffing against over-granular single keyword ad groups in Google Ads, using a Test, Kill, Push, BAU, Top workflow so only proven search queries get their own ad groups.
Ladder
It’s What’s Inside That Counts: Get Advanced Creative Insights via Spotlight
Why creative testing drives ad growth but is painful to report on, how tagging ads by their text and image assets enables multivariate analysis, and the beta launch of Spotlight, which tags creative automatically.
Ladder
Ladder Growth Blueprint: The Most Important Document In Marketing Strategy [Interactive Template]
Ladder's free interactive Growth Blueprint template, which maps funnel economics, compares paid and organic performance against benchmarks, forecasts growth, and keeps OKRs, tactics, competitor research and personas in one document.
Ladder
Marketing Analysis For Growth: A Step-By-Step Guide
A step-by-step guide to marketing funnel analysis that turns user data into clear answers for a CEO, built on the pattern of insight, context and action, plus benchmarking and segmentation.
Ladder
ICE Framework: Scoring Tactics Would Get You Killed On The Battlefield
An argument against ICE scoring for prioritizing growth tactics, drawing on Gary Klein's research into how experts make decisions, and favoring expert judgment, fast action and learning from what survives.
Ladder
A Smarter Way to Track Campaigns in Google Analytics (+ Free URL Builder)
How sloppy UTM tagging corrupts campaign data, ten rules for consistent link naming, and steps to build a custom URL builder in Google Sheets with concatenation and validation checks, plus a free template.
Ladder
The Art of the Deal: How to Become a Master at Business Negotiations
Negotiation advice that treats reputation and fairness as the long game, citing the ultimatum game, with tactics such as framing, mapping the territory, being willing to walk away and listening more than talking.
Ladder
Business Ideas: How to Test A Business Venture Before Quitting Your Job
A five-step method for validating a business idea before quitting a job: pre-sell it, map the strategy on a lean canvas with an elevator pitch, validate demand, promote it, and define the offer.
Ladder
The Ultimate Guide to Learn ‘Growth Hacking‘: Become a Growth Hacker, For Free
A free self-study curriculum of curated blog posts and videos for becoming a growth marketer, split into strategy topics and growth channels, designed to take about 15 weeks part time.
Ladder
Acquiring A Growth Hacking Agency: Ladder’s M&A Of Growth Engine
The story of Ladder acquiring the growth agency Growth Engine, including why the author founded Ladder and how the deal was tested first with trial retainers, shared training and a one-month review.
Ladder
The Harrison Ford Effect: Big Careers Start with Small Salaries [+ My Salary Diary]
Harrison Ford earned $10,000 for the first Star Wars film; the piece argues early careers should optimize for learning over pay, illustrated with the author's own job-by-job salary diary and Noah Kagan's.
Ladder
How to Build and Set Better Marketing Budgets
Why a marketing budget needs one priority rather than several, and a five-step method for realistic goals: research benchmarks, establish the counterfactual, tie goals to survival, carve out testing budget and estimate test success.
Ladder
Your First 10 Customers Should Be Won By Hand-to-Hand Combat
Advice for founders with fewer than 10 customers: skip ads and SEO and close each sale by hand through personal networks and direct conversations, since those first sales reveal what later marketing needs.
Ladder
Marketing Plan Template: A Successful Agency’s Marketing Plan Example
Ladder open-sources its own marketing plan, run on a $5k monthly budget, as a template covering brand positioning, market research, channel audits, a tracking plan, a tactic backlog and monthly results.
Ladder
Steve Jobs as a Service Doesn’t Exist. Stop Outsourcing Your Genius.
Drawing on 102 startup clients, 28 of which failed, an argument that founders cannot pay an agency to supply the product vision they lack, because outside help can only magnify what is already there.
Ladder
How To Advertise On Facebook: Demystifying How Facebook Ads Work
A simplified formula for how Facebook earns from ads (revenue per user equals CPM times number of ads) and what it implies for advertisers' bidding, budgets and conversion goals.
Ladder
Market Research: Ladder’s Competitors and Influencers
Ladder's research into competitors on both the software and agency side, including GrowthHackers Projects and HubSpot, plus a list of marketing influencers and advice to add value before asking them for anything.
Ladder
Ladder.io’s Target Audience for Our 2017 Marketing Plan
How Ladder researched its 2017 target audience with Facebook Audience Insights and website visitor data, replacing guesswork personas with data-driven ones and checking audience sizes before building campaigns.
Ladder
Ladder.io’s Marketing Strategy Executive Summary [Free Template]
Ladder's own marketing strategy executive summary, shared as a template: tagline, executive bios, founding story, mission, brand, SWOT, lean canvas, customer persona, competitors and elevator pitch.
Ladder
[SEO Audit] We Analyzed Our SEO. Here’s What We Learned:
Ladder's audit of its own site's SEO, organized around accessibility, relevancy and authority, covering robots.txt, sitemaps, speed, on-page optimization and backlinks, with a bonus social media audit.
Ladder
We Audited Our Email Marketing Campaigns. Here’s What We Learned:
Part four of Ladder's marketing plan series: an audit of its broadcast, drip and behavioral emails across MailChimp and Intercom, including the Gmail plus-address trick for signing up repeatedly to test flows.
Ladder
UX Teardown: What We Learned by Auditing Our User Experience
Part three of Ladder's marketing plan series: a teardown of its own website through free user tests, a walk through the conversion journey, and heatmaps suggesting the homepage is too long.
Ladder
The Growth Process That Drives Ladder’s Success
The sprint process Ladder uses to run growth experiments for clients, moving through analysis, strategy, stress-testing and approval, execution, optimization and Friday learning sessions, later stretched from weekly to monthly cycles.
Ladder
We Audited Our Analytics Setup. Here’s What We Learned:
Part two of Ladder's open marketing plan series: an 11-point checklist for auditing an analytics setup, from installed tags and conversion events to UTM naming and passing UTM data into the CRM.
Ladder
The Ultimate PPC Audit Walkthrough: A Step-By-Step Guide For PPC Analysis
Ladder audits its own Google Ads and Facebook Ads accounts with real data, using a five-step loop: segment the data, find best and worst performers, spot differences, research context, and recommend action.
Ladder
5 Cheap and Effective Ways to Run Facebook Ads A/B Tests
Five low-cost tricks for running Facebook ad A/B tests on a limited budget (observation, bandit, deprivation, geography and precision), paired with sample tactics such as loss-aversion ad copy and lookalike audiences.
Ladder
How to Forecast Performance for Your Marketing Campaigns
Why extrapolating daily averages gives poor marketing forecasts, and how plotting spend against conversions reveals diminishing marginal returns, a free spreadsheet version of the Efficient Frontier method.
Ladder
How We Hire A New Marketer Every Month (+Our Excel Test)
Ladder's process for hiring a marketer every month, assembled from other companies' best practices, with an application survey, a downloadable Excel test, a copy test, interviews and a paid half-day workshop.
Ladder
The Marketing Metrics Checklist: 8 Numbers You Need To Check Daily
A 30-minute daily routine for paid marketing: pick one of eight metrics, from CPA and CTR to maximum spend cap, and run five checks on it, framed by a story of learning math by osmosis.
Ladder
The Marketing Skills You Need to Become a Full-Stack Marketer
The skills a generalist, full-stack marketer should build first: Excel, blogging, talking to customers, basic HTML, CSS and JavaScript, and user experience, with data science and a growth process as advanced extras.
Ladder
App Store Launch Plan: How to Launch a New Mobile App
A three-phase plan for launching a mobile app: make sure it works, push for a burst of downloads all at once, then keep momentum, since App Store rankings reward download velocity.
Ladder
The Excel Tutorial For Marketers: Spreadsheet Functions Marketing Hires Need To Know
A downloadable Excel workbook and walkthrough of nearly 30 spreadsheet techniques marketers need, from paste special and VLOOKUP to SUMIF, text functions and pivot tables, built on real anonymized marketing data.
Ladder
Cascading Significance: How to Make Data-Driven Gut Decisions
Most marketing decisions lack the data for statistical significance, so the piece proposes Cascading Significance: start with a gut call, then build confidence over time by gathering data, with tips on modeling sample size.
Ladder
“We Have a 57% Failure Rate”: A/B Testing at Ladder
An analysis of 330 marketing tests Ladder ran for 16 clients, which found that 57% failed, and an argument that the answer is to lose small and win big by A/B testing every bet.
Ladder
Recast
Contributing writer · 2022–2023
Wrote about marketing mix modeling, incrementality testing and measurement for Recast’s blog.
Visit getrecast.comMER is a Road to Ruin
Argues that the Marketing Efficiency Ratio misleads scaling brands because revenue from other causes drives up ad budgets, and recommends triangulating with attribution, marketing mix modeling and lift studies.
Recast
The incrementality test that got my boss's boss fired
A story from early in the author's career when a cheap conversion lift test showed a display vendor's results were only 40% incremental, and the political fallout that followed.
Recast
Monte Carlo Simulations: A Tool for Predictive Analysis and Informed Decision Making
A beginner's guide to Monte Carlo simulations, how random sampling estimates the odds of different outcomes, and their role in Bayesian marketing mix modeling, with simple Python and R examples.
Recast
Measuring the Long Term Effects of Brand with MMM
Covers a GfK, Nepa and Meta approach to measuring long-term brand effects by modeling how ads drive brand awareness and awareness drives sales, then rebuilds it as a tutorial on simulated data.
Recast
MMM Template
A free Google Sheets template for building a simple marketing mix model by hand, with an explainer video, offered as a way to learn the method before moving to code.
Recast
Meta Sustainable Measurement Summit
Highlights from Meta's Sustainable Measurement Summit, focused on mobile gaming after iOS14, including a stat that 44% of advertisers named MMM their top fix for lost tracking.
Recast
Meta Marketing Mix Modeling Summit Highlights
Highlights from Meta's second marketing mix modeling summit, where Meta reported 80% more MMM adoption, covering sessions on calibration with lift tests, creative effects and long-term brand measurement.
Recast
Your email open rates aren't as good as you think they are
How Apple's Mail Privacy Protection preloads email images and inflates open rates, and what to do about it: rely on your own benchmarks, design for anonymous readers and build owned channels.
Recast
Google's Data-Driven Attribution: a $40,000 mistake
An agency story about switching a Google Ads account to Data-Driven Attribution, which over-credited display conversions and led to a $40,000 client refund, and a case for spend alerts and independent attribution.
Recast
The difficulty measuring 'dark social' traffic and word of mouth
Why word of mouth shared through private channels shows up as direct traffic, and a set of methods for estimating it, from tagging links and surveys to the Word of Mouth Coefficient and marketing mix modeling.
Recast
Limitations of GeoLift Experiments
Explains how geographic lift tests measure incrementality and walks through their limitations, including contamination between regions, saturation, delayed effects and outliers, plus tips on matched markets and power analysis.
Recast
Cleaning URLs for Accurate Channel Attribution
Explains how shared links carrying UTM parameters give the wrong channel credit for later visits, and offers a short JavaScript snippet that strips the parameters after analytics records them.
Recast
How much data do you need for your Marketing Mix Model?
Uses simulated data and parameter recovery in a spreadsheet to test how much data a marketing mix model needs, finding accuracy holds with little data while channel estimates suffer.
Recast
How Uber Orbit Works
A review of Uber's open source Orbit library and its Bayesian time-varying coefficients for marketing mix modeling, with notes on what it lacks, such as saturation curves and a budget optimizer.
Recast
How Google LightweightMMM Works
How Google's open source LightweightMMM library builds a Bayesian marketing mix model in Python, covering run time, adstock, seasonality and geo support, and how it compares with Meta's Robyn.
Recast
How to make a marketing report that accounts for macro trends
A pointer to a Funnel.io guest post showing how to pull ad platform data into Google Sheets and run a linear regression for marketing reports that adjust for macro trends like COVID.
RecastIntro on Recast, full post on Funnel
Setting Realistic Expectations Between Marketing and Finance
A pointer to a Causal guest post on getting finance to approve marketing budgets by forecasting a range of outcomes instead of a single number.
RecastIntro on Recast, full post on Causal
Creative performance drives 47% of sales: why is it missing from most models?
Starts from a Nielsen finding that creative drives 47% of campaign sales, explains why most marketing mix models leave it out, and outlines three ways to include it: scorecards, message mix modeling and hierarchical models.
Recast
Negative ROMI: Lose money to make money
Why a negative return on marketing investment can be the right call for growing companies, using unit economics, economies of scale and lifetime value, with examples from DoorDash, Netflix and Amazon.
Recast
Decomp RSSD: Optimizing for Politics
Weighs the case for and against Decomp RSSD, the metric Meta's Robyn uses to favor models that agree with your current budget, and suggests Bayesian priors as a better way to add business logic.
Recast