Subject line testing tends to use the formula below.
⚖️ Pick two subject lines and test them with a sample of your email list.
⏳ Wait a set amount of time. Then whichever subject line has the higher Open Rate is the winner.
? Send the winning subject line to the remaining subscribers.
The definition of success for this method depends on Open Rate.
But, Open Rate is unreliable.
Wait, what?
Open Rate is an unreliable metric because it depends on your Email Service Provider (ESP) being able to accurately identify users that open your emails.
“My ESP does that. It shows me Open Rate and shows me Opens in my dashboard,”
? Yes it does, but it might not be accurate.
An Apple update back in ~September 2021 rolled out Mail Privacy Protection (MPP) alongside iOS 15.
If a user enables Mail Privacy Protection, ESPs no longer can accurately identify opens.
This is because Apple preloads content and data from your ESP. This includes tracking pixels which power your ESP’s dashboards and generates Opens, Open Rate, etc.
This means a subscriber with MPP-enabled could have a 100% Open Rate due to Apple preloading the email and triggering the tracking pixel.
When you conduct a subject line test with the method mentioned below, you’re relying on being able to accurately identify an Open or calculate Open Rate.
Emails sent to users with MPP-enabled will always be counted as “opened”.
This makes the method unreliable.
There are other ways to increase your Open Rate:
- Try a 50/50 approach. Send Subject Line A to 50% of the list and Subject Line B to the other 50%.
Yes, send the campaign to the entire list and analyze post-send.
You don’t pick a winner based on the first X% that open because there’s a high probability that the opens being counted are not real opens.
If your ESP provides this functionality, you could export the testing data, remove Apple devices, and analyze the different cohorts.
But, besides testing subject lines…
Try and think of metrics beyond Open Rate.
? Conversion rate.
? Growth rate.
? Revenue.
? Click Rate.
? Referral rate.
? Unsubscribe rate.
It’s important to think of each email you send with a goal in mind. Do I want to increase sales for my course? Do I want readers to engage with my content? Do I want readers to send referrals to my newsletter?
With your readers in mind, Opens and Engagement will come.
And remember, each audience is unique.
What one company does and applying it to your own won’t always yield the same results.
Take it as an opportunity to test hypotheses with your own audience.
submitted by /u/newsletterxyz
[link] [comments]
