What 120,000 randomized recipients say about a first name

Ask whether personalizing a cold email gets more replies and the answer arrives as a percentage lift, usually a large one, usually drawn from a sending tool's own campaign data. A number built that way has a problem inside it. Senders who fill in merge fields are also the senders who research a shorter list and write a better email, so the lift carries all of that with it. Randomized trials have asked the narrower question, where the only difference between two emails is the personalization itself. Their answer is smaller and more useful.

The pooled answer is about a quarter

The largest set of randomized trials on whether personalizing a message increases responses sits in medical research methodology rather than marketing. Cochrane's review of methods to increase response to postal and electronic questionnaires, updated in November 2023 by Edwards and colleagues, found 88 eligible trials covering more than 30 ways of raising response to electronic questionnaires. Twelve of those trials, 48,910 people between them, compared personalized electronic contact against standard contact. The pooled odds ratio is 1.24, confidence interval 1.17 to 1.32, so personalization raised the odds of a response by roughly a quarter and the interval sits clear of no effect. In a review where most larger electronic findings rest on one or two trials, that is unusually solid ground.

The commercial experiment, in absolute numbers

The strongest commercial evidence comes from randomized field experiments by Navdeep Sahni, Christian Wheeler and Pradeep Chintagunta, published in Marketing Science in 2018 and readable as a Stanford working paper. Their main partner sold certification training to corporate addresses and counted a sales lead as a reply with intent to buy after seeing the price. They randomized 68,088 addresses into two groups whose emails were identical except that the treatment subject line began with the recipient's first name. Opens went from 9.05 percent to 10.80 percent. Leads went from 0.39 percent to 0.51 percent. Unsubscribes fell from 1.20 percent to 1.00 percent.

That lead movement is the 31 percent improvement often quoted from this paper, and it is real and carefully measured. In absolute terms it is four leads per thousand emails becoming five. Worth having, and worth knowing the size of before anyone builds a quarter on it.

What the name is actually doing

A later experiment in the same paper is the one worth reading twice. By then every subject line carried the name as standard, so the researchers varied two things inside the body across about 1.4 million emails: whether the name appeared again, and whether the email mentioned a possible discount. The name alone did nothing measurable to leads, at p equal to 0.38. The discount alone did not reach significance either, at p equal to 0.27. Together they did, raising leads from 0.021 percent to 0.032 percent at p below 0.01. The authors read this as attention: seeing your own name makes you read the rest more carefully, so whatever else it says counts for more.

The name persuaded nobody. It got the sentence next to it read.

That says what a merge field is for. Personalization behaves like a multiplier on the substance of the email, and a multiplier applied to nothing stays nothing. If the body offers to connect and explore synergies, the name in front of it mainly raises the odds that someone reads you saying so.

Where a heavier dose stops helping

Tekila Harley Nobile and Lorenzo Cantoni, writing in Digital Business in 2023, ran the one field experiment that varies the dose. A retailer sent 2,303 triggered emails at three levels, from no placeholders up to name, address, last purchase date and images of the product the person had searched for. Opens were highest in the middle: 61.5 percent at the low level, 67.0 percent at medium, 62.7 percent at high. Purchases among the people who clicked ran 43.9 percent, then 42.8 percent, then 29.5 percent at the heaviest level, a fall their model found significant at the five percent level. Their reading is that the heaviest version came across as pushy. Pointing the same way, Jesús Trespalacios and Ross Perkins assigned 1,598 students to four versions of a survey invitation and report that neither the degree of personalization nor the email's length affected response or completion.

Most of this is measured on an open

A caveat runs under the whole literature, and the Sahni paper names it in a footnote on its own method. Opens are counted with a tracking pixel, which makes the measure noisy, dependent on the recipient's software, and incomplete, because some clients never load the image. Their later experiment reports an average detected open rate of about 3 percent and calls it exactly that. So part of what moves in an open-rate finding may be which software fetched an image. The results that survive the objection are measured further down the ramp: a lead, a purchase, a reply. That is why Fast Mail Tracker ranks signals as sent, seen, opened, clicked and replied, and records a privacy relay prefetch or a datacenter scanner with a label instead of a confirmed open.

What to check before you send

• Look at what your merge source actually holds. Fast Mail Tracker substitutes name and email placeholders into each copy in a campaign, and a row with no name falls back to the part of the address before the at sign. A thin row greets someone as j.smith rather than leaving an obvious blank you would have caught.
• Spend the effort on the sentence after the greeting, since that is what the name buys attention for.
• Stop adding placeholders once they stop adding information.
• Judge the change on replies rather than opens, and give it enough volume to mean something. These trials needed thousands of messages to resolve effects this small, so a gap across fifty sends is noise wearing a result's clothes.

None of this makes personalization optional. It makes it a lever with a known size and, better, a known mechanism, which is more than any lift figure offers. Put the name in, keep the merge source clean, then spend the real work on the sentence it buys a reading of. To watch what that does to your own replies, Fast Mail Tracker is in early access while Google's security review runs, and the install button below this article is the way in.