Download All Your Emails

[This part 2 of a multi-part series – start here.]

Before we purge our online email history, we should make a complete and searchable backup.

Blazing Trails

Email apps, or “clients”, are designed primarily to send and receive emails. They also have the ability to store older emails, including searching them, organizing them into folders, adding tags, and so on. But if you have lots of old emails, there’s a good chance that your older emails aren’t really downloaded. To save disk space and internet bandwidth, many email clients will only fetch the metadata for your older messages – the date, time, to, from and subject. You’ll see a placeholder in your inbox view, but the contents will only be fetched (on demand) when you try to read it. So we’re going to have to do some special things to ensure we get complete local copies of all emails still available from your provider.

As I said in the first installment, I’m going through this process myself, taking you along with me. As such, I’m learning lots of things as I go. There’s a very clear way that this process should work… and then there’s the sometimes-clunky way that it actually works. It turns out that email providers don’t really have an incentive to make this easy. But in their defense, most people don’t archive emails like we’re about to do. But I’m blazing the trail for you. I’ve already gone down a couple paths that didn’t work.

Email Download Strategy

Here’s our strategy, at a high level. This will work for just about any major email provider, though we’re going to focus here on Gmail (Google), Yahoo Mail and Outlook (Microsoft). The email client I’m going to recommend works on Macs (macOS) as well as PCs (Windows).

Email Retrieval Mechanism: IMAP

When you open up your email client, like Apple Mail or Microsoft Outlook, it fetches emails from your provider using a standard protocol called Internet Message Access Protocol (IMAP). Because it’s a well-known, common standard, it will work for just about any provider.

I was originally hoping to use modern privacy dashboards (thanks mainly to laws like GDPR) to download a usable archive of my emails. Google does allow you to download an email archive in MBOX format (one massive file) via their excellent Google Takeout tool. However, my chosen email client does not (currently) have native support for ingesting this archive.

My plan here is to have a dedicated email app for accessing your email archive. I don’t want to use this app for any other purpose. Why? Well, most email clients are meant for sending and receiving. If you have a single app that is actually used for sending and receiving emails on other accounts, I think it could get confusing. Also, if you’re going to leave your Gmail, Yahoo Mail and Outlook accounts active (and I recommend you do), then you would need to somehow separate your archived emails from the real, live emails you may continue to receive from those accounts.

Email Archive Client: Thunderbird

There are many nice email clients out there, and you’re welcome to find one you like better. However… the one I’m going to recommend works well for downloading full copies of all emails, it’s been around for years and has a lot of support, it’s free and open source, and it runs on both Mac and PC. And I also think it’s likely that most people are not already using this app, meaning that it can be dedicated for this one purpose. (If you have a Windows PC, you might want to take a look at Mailstore Home. It’s free and I’ve heard good things. If they had a version for macOS, I probably would have gone with this app.)

I chose Mozilla’s Thunderbird. You can download the app here. It’s not in the Apple or Microsoft App Store, so when you install it, you may get a warning about opening it the first time. As long you as get it from the link I provided, you’re good.

So to the point about MBOX support, Thunderbird requires a plugin to import this type of archive, and it’s an ugly, scary process. And since Yahoo Mail, Outlook and others can’t use this technique (they don’t currently offer a way to get your archive in this format), I decided to stick with the IMAP import method.

A Note About Email Account Access

Setting up Thunderbird (or any email client) to access your online email, you will need to log in to your account. Exactly how this works varies – from service to service, and even over time. I’ll do my best to point you at all the resources you might need, but I can’t document them all in the article or it would be massive.

You may need to have two-factor authentication (2FA) enabled before you can connect to your account from the email app. You probably already have this set up – if you’ve ever been texted or emailed a 6-digit code to log in, or had to enter a code from an authenticator app, then you’re set. If you don’t think you have 2FA enabled or want to verify it, see these articles: Gmail, Yahoo Mail, Outlook. Honestly, if you aren’t using 2FA, it adds a lot more security – I highly recommend it.

There are two account setup methods you may encounter when adding your email accounts to Thunderbird: an app-specific password or web (OAuth) login. The latter method is much simpler – you’ll be presented with a pop-up window asking you to log in directly to Google, Yahoo or Microsoft, and you log in normally. However, in my experience, Thunderbird often failed with a “400” error during this process. (Update: it seems that setting Thunderbird to allow all cookies will fix this error.)

Thunderbird Setup

First, we have to set up Thunderbird on our computer (you can’t do this on a phone). Depending on how much email you have, you may need several gigabytes of free storage space. If you have the option, I would do this on a desktop computer, but a laptop will work, too. We’re basically going to configure Thunderbird to download full copies of all your emails and index them so they can be more quickly searched and filtered.

1. Download Thunderbird here and install it on your computer.
   • If you already use Thunderbird for email, I would highly recommend creating a separate profile for your email archive.
2. Close the “hub” dialog on startup. We need to change some settings before we add any accounts.
3. Open “Settings: General” (not “Account Settings”):
   • Turn off ‘Always check to see if Thunderbird is the default…’ (unless Thunderbird is already your email client)
   • Turn ON ‘Allow Spotlight to search messages’ (Spotlight is Mac only; Windows may have a similar option for indexing – turn it on)
   • Turn ON “Enable Global Search and Indexer”
   • Set “indexing” (all the way down in “Network & Disk Space”) to “file per message (maildir)”.
   • In “Offline” settings (which is a button labeled “Offline…”)
     • turn ON (“Yes”) “Download messages for offline use”
     • turn OFF (“No”) “Send unsent messages when going down” – we won’t be sending messages from here
4. Open “Settings: Privacy & Security”…
   • IMPORTANT: Turn ON “Accept cookies from sites” and set “Accept third party cookies” to “Always”. Normally I would say not to do this, but you need this enabled for the account login to work properly (which uses OAuth).
   • Turn ON “tell websites not to sell or share my data” – always good policy, though we may never need this
   • Remote content: turn OFF (you can override this on a per-email basis if you want)
   • Telemetry: turn OFF
   • DoH: bump to Increased Protection

Connect Thunderbird to Your Email Service

This is where we actually connect to Gmail, Yahoo Mail and Outlook to download your emails. This will retrieve copies of all your emails – they will still be available on the email provider’s server, as well. That means you can still read those emails using whatever app you were using before. But the plan will eventually be to delete them on the server, leaving only your local copies. Don’t worry, I’ll help you make secure backups, just in case.

Go through this process for each email provider you’ve used, even in the distant past. I have all three, personally – you may have only one or two.

• Gmail (Google)
• Outlook (Microsoft)
• Yahoo Mail

If you find any errors or confusing instructions, please let me know.

Next Step

In the next article, I’ll tell you how to verify your email archive and give you another interesting backup method you might want to try.