How to set up a mail server on AWS EC2 using open source software

When running your own mail server you have complete control over its performance and reliability. Here is how to get started with an open-source mail server on AWS EC2.

Running a mail server on AWS EC2 is not quite as easy as it sounds.

1. Register your domain name

Register your domain name with a trustworthy registrar. The company that performs this service is now officially your domain’s registrar. It is now your responsibility to keep your contact details with this company current at all times. If you don’t want to disclose so much personal information, you can buy a domain privacy service in most jurisdictions, but you still have to maintain your account with your registrar in good standing or you risk losing your domain. This is where using a respectable, trustworthy registrar pays off: they will go the extra mile to alert you of any bumps in the road and protect you from potentially disastrous events such as 3rd party’s attempts at domain hijacking.

After registering your domain name you can configure the routing.

2. Get your NS servers from Route 53

https://console.aws.amazon.com/route53/

Create a new Public Hosted Zone using the domain name from step 1. This will reserve four of Amazon’s name servers for the domain of your choice (the minimum required are two but you should use all four). Make a note of their host names; you don’t need to know their IPs.

For more on DNS configuration see this post: A quick guide to DNS configuration using AWS Route 53 (or any other DNS service)

3. Assign your domain’s name servers to your registrar’s record of your domain

Your registrar might have already assigned their own name servers to your domain. This is a common practice with services such as domain parking, where you aren’t yet hosting a website but merely keep the domain in your account. Whatever the case with your registrar’s default ns servers, in most cases you will want to exchange them.

Sign back into your account at your registrar and change your domain’s default name servers to your new name servers which you acquired via AWS Route53 in Step 2 above.

4. Edit your hosted zone to set up NS routing

At this point, there are two possible scenarios. You can:

set up a mail server from scratch
use an existing mail server that has been pre-configured on another domain and is already up and running.

In the first case, you need to create an A record and an MX record in the Hosted Zone of your domain. The A record should point to the IP address of your EC2 instance that will host your mail server. The MX record defines your mail exchanger. This is the host that runs your mail transfer agent (for example Postfix).

In the second case, where you merely add your new domain to an existing setup, all you have to do in terms of the ns configuration is create an MX record that points to the host which is running a fully functional mail server. The default priority of 0 is meaningless with only one mail host.

Within your domain’s Hosted Zone, click on the button Create Record Set and assign it a name of your choice (typically mail.domain.tld or smtp.domain.tld). Select the type “MX — Mail exchange”. No, it’s not an alias. Keep the TTL value at its default. Enter the host name of your mail server host with the default priority of 0 in the form:

0 smtp.somefancymailserver.tld.

(with the dot at the end). A lower number means a higher priority. Set the routing policy to Simple and click on Create to confirm. The result should look something like this:

smtp.yourdomain.tld. MX 0 smtp.somefancymailserver.tld.

After a time span defined by the TTL value, the routing should be fully operational but may not have propagated yet. Patience is the key to success here. In the meantime, you can use a service such as intodns.com to verify your configuration.

5. Request a reverse DNS record from AWS

Request a reverse DNS record from AWS using this form (AWS root access required):

https://aws.amazon.com/forms/ec2-email-limit-rdns-request

6. Configure your MTA

The specifics of this step depend on your choice of an MTA. One of the best open source MTAs is Postfix, of course. It supports several types of mail users:

Unix system users (such as root or postmaster)
vmail users: these users access virtual mailboxes using canonical (“real”) or hosted domains (so-called because they are not canonical to the Postfix host)

You can implement hosted domains in one of several ways, namely as:

virtual alias domains: mail users on these domains are always aliases of addresses in other domains (these can be local UNIX system accounts or remote addresses); to configure these accounts, use the virtual_alias_maps parameter
virtual mailbox domains: each mail user can have its own mailbox and is not required to have a UNIX system account; to configure these accounts, use the virtual_mailbox_domains parameter

Your domain (yourdomain.tld) will be a hosted domain in this configuration.

First, Postfix must recognize your hosted domain as legit. In order for Postfix to take over the responsibility for handling mail for it, use the parameters virtual_alias_domains and virtual_alias_maps.

When changing Postfix map files, remember to execute postmap on them and to reload Postfix. After changing configuration files, remember to restart Postfix.

Once you complete this setup, there is only one more major challenge to overcome and it has to do with reading mail.

7. Configure an MDA

And MDA is the software on your mail server host that delivers messages to your mail client on any device of your choice. One of the best open-sourced MDAs out there is Dovecot.

Assuming a working Dovecot configuration on an existing mail server, there is only one more action you need to perform: create access credentials for a Dovecot user on your new domain. The most typical setup consists of a user name (typically the email address) and a password. Hash the password using:

doveadm pw -s SHA512-CRYPT

Save the hash—not the password!—in Dovecot’s users database (search for the parameters passdb and userdb), then restart the service. This is how the password hash looks in the config in /etc/dovecot/users:

you@yourdomain.tld:{SHA512-CRYPT}alongstringofrandomlookingcharactersfollowedbyfourcolons::::

Verify authentication

If you are having trouble connecting to Dovecot remotely, try the command line.

Check if the user exists:

doveadm user sales@cloudinsidr.com

Verify authentication:

doveadm auth you@yourdomain.tld yoursecretdovecotpassword

If you succeed, you know that your credentials work with Dovecot. Your mail client, when configured properly, should have no trouble connecting to your EC2 instance running Dovecot to retrieve the mail that Postfix stores there.

In case you need some more verbose debugging information, open the file

/etc/dovecot/conf.d/10-logging.conf

and activate:

auth_debug_passwords = yes
auth_debug = yes

Restart Dovecot, repeat recent activity and check your logs:

grep -i username /var/log/dovecot.log

7. Configure your mail client

Last but not least, you need a software to handle email on your end device. Microsoft’s Outlook, Postbox, Thunderbird and Apple Mail are some popular choices.

Once this setup is completed, your administrative duties will only begin in earnest. Spam is the name of the game in mail server administration.