Create your own mail server on EC2


[Update – April 16th 2010] As of April 15th 2010 the version ClamAV used as the the virus scan agent in the mail server built using these instruction will nolonger update the virus database which means the virus scanner will no run and causes the mailserver to defer send emails. As a result the ClamAV modules MUST be upgraded. When you know how to upgrade this is straight forward. See the instructions in this post for more information.


[Update – May 14th 2010] Maybe it was a setting I changed by accident or maybe the email server image I used was configure this way, but the combination of Amavis and Postfix has led to a lot of backscatter. The good news is that I’ve found out how to cure the problem. The file /etc/amavis/conf.d/20-debian-defaults contains the setting uses to control whether or not mail detected as spam or that potentially contain viruses. Here’s how they used to look (bouncing lots of emails causing backscatter)
<br />
$final_virus_destiny      = D_DISCARD;  # (data not lost, see virus quarantine)<br />
$final_banned_destiny     = D_BOUNCE;   # D_REJECT when front-end MTA<br />
$final_spam_destiny       = D_BOUNCE;<br />
$final_bad_header_destiny = D_PASS;     # False-positive prone (for spam)<br />

Here’s how I think they should look (now discarding banned and spam emails):
<br />
$final_virus_destiny      = D_DISCARD;  # (data not lost, see virus quarantine)<br />
$final_banned_destiny     = D_DISCARD;   # D_REJECT when front-end MTA<br />
$final_spam_destiny       = D_DISCARD;<br />
$final_bad_header_destiny = D_PASS;     # False-positive prone (for spam)<br />


As a result of finding out that our emails supplier may be bouncing valid emails just because a sender’s DNS settings are not just-so we decided to create our own mail server. It was surprisingly easy and, for us, cost effective.

We’ve created a server using a virtual machine on Amazon’s Elastic Compute Cloud (EC2) which took a day and that included learning about the mail server software. The mail server includes anti-spam measures, control over STMP attacks, support for white/grey/blacklists, web mail, security using SASL (and the EC2 firewall feature).

Thanks go to Ivar Abrahamsen (Flurdy) for this article showing how to install and configure all the components needed for a good quality mail server. He also deserves a mention for then creating Amazon Machine Images (AMIs), based on the Ubuntu AMIs created by Eric Hammond, with all the goodies already installed.

Economics
The cost of running a mail server will depend upon how you go about it. We have a lot of experience using Amazon’s EC2 so using the pre-built AMIs created by Ivar made a lot of sense. It makes sense economically too. The standard price of a small Linux is $0.08/hour but can be run using ‘spot prices’ for $0.05/hour which is roughly $30/month. Even a small machine on EC2 will provide support for all our staff. There is no marginal cost for adding more than one account so it’s allowed us to add email accounts for all our domains – something that’s not been cost effective so far.

Compare this with, say, Google where its $50/year. As soon as we have 7 accounts setup we’ve broken even (and we have many more than 7). Plus we can also add accounts for all our domains. Because we control the whole machine and are not just renting a service, it can be used as another backup for our web site or provide some other useful service for us.

An economic aspect not considered here is the cost of learning to use Linux. We do know how so its not an issue for us. If you don’t it’s not going to be as straight forward or cheap. Likewise we’re long time users of Amazon Web Services so there’s no learning cost involved here either.

Getting started

Ivar’s article covers the points but I’ll reiterate that you will need to:

1) Create an Amazon Web Services (AWS) account if you don’t have one already.

I’ve found Amazon’s ElasticFox add-in for FireFox a really easy way to manage EC2 virtual machines and resources though the AWS console also provides GUI-type control over your EC2 machines (there’s a whole command line interface and an API but best to keep away from that).

S3Fox is also great for managing Simple Storage Service (S3) resources. S3 is a cloud storage service which along with EC2 is a service of AWS (those acronyms coming thick and fast) and is where EC2 stores your customized machine images.

If you have not used any Amazon Web Services before, the PDF on the Elastic Fox page shows how to use Elastic Fox to create new machines, configure and add hard drives, setup fixed IP addresses, configure firewall security.

2) Create a virtual machine using the ami-395fba50 AMI

This image includes: Ubuntu + Postfix + Courier IMAP + MySQL + Amavisd-new + SpamAssassin + SASL + TLS + Postgrey

(for the uninitiated, PostFix is the SMTP server, SpamAssassin is the email anti-spam component, Amavis links PostFix and SpamAssassin, SASL is the Secure Socket Layer service for security while TLS is an other security layer used in email clients like Outlook)

3) Make changes to configure the server for your needs.

The AMI almost works out of the box. The services will start but there are some changes that are needed. These changes are documented in Ivar’s but here’s quick summary of the changes we made:

a) Set the root password

AMIs do not normally have passwords because console access is via SSL and authenticated by PKI keys. However other applications may need a root password.

b) Set a fixed ‘Elastic’ IP address.

Use Elastic Fox to claim a fixed IP address and attach it to the running machine. This will allow the address to be setup in you DNS and ensure external mail transfer agents can find the mail server. This can also be done using the ECS console.

c) Edit the /etc/mailname file to add the EC2 public address.

It will be something like ec2-75-101-128-50.compute-1.amazonaws.com This address will be the address used in in the MX record in DNS

d) We couldn’t find the root password for the MySQL database.

The root password can be reset by using the instructions on the MySQL site. We found the alternative of using the –skip-grant-tables to be the most effective.

e) Find the password for the mail database (maildb)

PostFix is configured to use a MySQL database to hold mail domains, accounts and addresses. The database is called maildb. The username and password are ‘mail’ and ‘mailpassword’ respectively.

f) Disable PHPMyAdmin for managing MySQL

PHPMyAdmin is installed but there are better options. It’s installed on port 80 which means if you want to use the machine as, say, a web server the MySQL instance is easier to compromise.

g) Install Webmin

Webmin is your friend, especially if you are not a command-line file editing type. Webmin provides a web page way to manage much of a web server – including MySQL. Although it is also used over the web, it is installed on port 10000 which means the port can be made non-public.

To install webmin type the command:

apt-get install webmin

If it fails because packages are missing, type the command:

apt-get update

That’s it, should be installed. Now you can access Webmin at:

https://.amazonaws.com:10000/

when you will be prompted to enter a root user name (root) and password (see a)

h) Install POP3

By default only IMAP support is installed. Some email clients support IMAP (Outlook for example) and it provides more than just an inbox. However most have a local inbox and just want to be able to download inbox contents and send emails. So it’s probably useful to install POP3 support.

apt-get install courier-pop

and/or

apt-get install courier-pop-ssl

i) Set SSL keys

Read Ivar’s article about usingf openssl (which is installed) to generate 1024 bit keys for authentication. You will also need to generate keys for IMAP-SSL and POP3-SSL (see /etc/courier)

j) Disable Postgrey

Postgrey checks sender email addresses with spamhaus.org. Unfortunately *all* EC2 IP addresses are black listed. So you can apply to have you IP address removed or just disable Postgrey. This will mean you can send test emails to the mailserver.

To disable Postgrey switch off the postgrey edit /etc/postgrey/main.cf and remove:

check_policy_service inet:127.0.0.1:60000

from the smtpd_recipient_restrictions line.

k) Use Webmin to add domains, users and addresses (aliases) to maildb

Just as it sounds. Think it through and add what you feel is necessary.

As it turns out, what we really want is a relay server, a mail server that is public facing, will scan emails for spam/viruses and then forward valid emails to our in-house mail server (Exchange). This is even easier. All that’s necessary is to add:

relayhost = [myinternalserver.domain1.hdl]
relay_domains = domain1.hdl domain2.hdl

(The square brackets around the relayhost tell PostFix to *not* lookup the specified server’s MX record).

l) Now stop and restart the services

All the services are controlled by init.d. You can use the command line to issue commands such as

/etc/init.d/postfix restart

But it may be easier to use the System/Bootup and Shutdown page of Webmin when any or all services can be stopped, started or restarted.

m) Ask Amazon to disable SMTP throttling

I don’t know to what extent SMTP traffic over EC2 is throttled but whatever it is it can be disabled by applying to Amazon.

n) Set the EC2 securty groups to open only port 25 publicly

Securty groups can be created restrict access by port. In our case only port 25 (SMTP) is public. All other ports are either closed or can only be accessed from our corporate IP address. The ports accessible from our internal IP address are 22 (SSH), 110 (POP3), 443 (IMAP), 995 (POP3-SSL), 3306 (MySQL) and 10000 (WebMin).

4) Bundle the configured image

Now the server is configured, bundle it to create a private AMI you can reboot in case the machine terminates for some reason.

The tools to bundle a machine are included in all AMIs. Before you start you will need to copy your public and private keys onto the AMI. We put them on the /mnt device as cert.pem and private.pem respectively. These will be used in the bundling commands. You will run a set of commands like the following:

cd /mnt
mkdir ami

ec2-bundle-vol -d /mnt/ami -k /mnt/private.pem \
-c /mnt/cert.pem -p mailserver -r i386 \
-u –no-inherit

(The –no-inherit removes the embedded data belonging to Ivar. Without this option the command will fail.)

ec2-upload-bundle -m /mnt/ami/mailserver.manifest.xml \
-b mailserver \
-a \
-s

(This step uploads the AMI parts to S3 for persistent storage into a [-b] bucket called mailserver with a manifest file of mailserver.manifest.xml)

The final step is to use Elastic Fox or AWS Console to register a new AMI. In Elastic Fox you press the green ‘+’ icon and enter the name of the upload image in the format /. In the case of this example, mailserver/mailserver.manifest.xml

That’s about it – you should now have a working server and saved. Well, I say that’s it. That’s it except testing!

Information and Links

Join the fray by commenting, tracking what others have to say, or linking to it from your blog.


Other Posts

Write a Comment

Take a moment to comment and tell us what you think. Some basic HTML is allowed for formatting.

Reader Comments

[…] Originally posted here:  Dev Blog – » Create your own mail server on EC2 […]

Hi folks,

Great article. One thing I didn’t notice was any mention of EBS. Do you have your mail directories backed up to another server, or are you persisting it to S3?

I’m just curious how you’re dealing with the possibility of the EC2 instance failing.

Thanks,
Dan

Hi Dan

For now we’ve just bundled the image so in effect, yes, for the moment we’re persisting to S3.

ami-395fba50 is based on Aelastics Ubuntu Hardy and predates the ability to boot from EBS so that one’s out – at least until there’s an Ubuntu based on one of the newer kernels and we can find the time to create a new machine.

I the mid-term we’ll do what we do for our web site: add an EBS volume and move the log files there.

However in the event of a failure its only the mail log files we’d lose so we’re not rushing to change anything.

Our email service used to be run by a company called Easyspace and these servers are still there (it was free) – just with lower MX record priorities. So in the event of a failure, mail will still be collected because senders will pick the next server in the MX list.

Sooner of later we’ll realise the issue and reboot from our custom image.

Hi Bills,

I’m no mail expert, but isn’t it possible that you could lose mail with this strategy? Or is the EC2 server just an SMTP server, and not an IMAP server?

IF it was an IMAP server, couldn’t you lose drafts of messages, or have a message delivered successfully, then have failure and lose the message?

Again, I’m not a mail expert, but both of those seem like possibilities.

What am I missing?

Thanks.
Dan

Ah, see what you are saying.

However we’re using this as a public relay server. All email is checked and sent directly to our in-house server over a secure connection.

Yes, if we were using it as a mailbox store (which the server is capable of being) we’d have to store the mailbox folders on something like as EBS device as you suggest.

Hi, i am thinking about transfering my stuff to EC2.
My main consern is the mail and the availability of the EC2 instances.

I have people that rely on that 24/7 so EC2 seamed to be interesting with the scaling part but the fact that an istance can be brought down at any moment it scares me a bit.

thanks for the article, i,ll poke around for that and maybe transfer some of my personal stuff on there.

Charles,

I think your fears are misplaced.

We’ve been running our web servers and VoIP exchange on EC2 for 2 years without a hitch – this blog is on a server hosted on EC2. The mail server has been running for a month now. We started the mail server using spot pricing and expect that, sooner or later, it will switch of and we’ll need to start it again.

But that’s our choice. We’ve chosen to take the risk that on a particularly busy day spot prices may spike and we’ll need to restart this server. At that time we’ll decide whether to continue with spot pricing or pay the ‘retail’ price – as we do with our web servers – when the service is not affected by price.

No hosted service is 100% reliable and I’m sure we’ll face some problems but we’ve not faced them in 2 years. By contrast our in-house servers are much less reliable.

I’ve heard about AWS outages but I think they’re a bit of an urban myth or may one that affects S3 more than EC2 because, as written, these outages have not affected us.

Of course there have been times when one or more of the web servers have been unavailable but these issue s have always problems of our own making (for example by screwing with the OS)

I can only report what I know and I hope you do move to EC2 and enjoy the benefits we have. We’ve been able to cut our hardware budget *and* have a more reliable service with better backup.

Bill

Hi Bills!

Thanks a lot. This article helped me a lot. Just an nfo:
The mysql root password is rootPASSWORD (For the ones who couldn’t find or couldn’t reset (like me) )

Cheers..

The following does not work, with either the ami you mention nor the latest ami-c0ee06a9.

To install webmin type the command:

apt-get install webmin

If it fails because packages are missing, type the command:

apt-get update

$ cat /etc/apt/sources.list
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid main restricted multiverse universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid main restricted multiverse universe
deb http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main restricted multiverse universe
deb-src http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ lucid-updates main restricted multiverse universe
deb http://security.ubuntu.com/ubuntu lucid-security main restricted multiverse universe
deb-src http://security.ubuntu.com/ubuntu lucid-security main restricted multiverse universe

We’ve not experienced this issue so thanks for sharing. Maybe the sources.list has been changed somehow (or webmin moved).