Scalable mail storage with Dovecot and Amazon S3

Storage Space

S3 Load Balancing

The road to a finished Ceph cluster with the S3 front end is actually not as rocky as it looks at first glance. Inktank [4] has a powerful tool up its sleeve in the form of ceph-deploy, which Ceph provides on systems with common distributions in just a few minutes. Anyone who first wants to test on three virtual machines can, of course, do just that.

Ceph is now integrated in common automation tools like Puppet or Chef. The Ceph cluster itself is therefore quick. Just a little more overhead and the Ceph Object Gateway is connected: Inktank itself provides detailed documentation that explains the entire process [6]. In the end, you have an instance of the Ceph Object Gateway that speaks S3 and can be used as a starting point for a Dovecot instance.

You should, however, bear in mind that the S3 gateway can break quite a sweat with large setups. No wonder: If there is only a single gateway instance, all connections from the Dovecot front ends to Ceph have to squeeze through the eye of a needle. This is not just a problem from the perspective of performance, but also a classic single point of failure (Figure 6).

Figure 6: A simple Ceph gateway setup: The single gateway instance, however, is a single point of failure.

However, because S3 is based on HTTP and there are enough high-availability approaches for HTTP, the solution for the problem is obvious: It makes sense to start with two object gateways, for which a (highly available) load balancer (LB) redirects incoming connections accordingly (Figure 7). This architecture is also scalable; additional object gateways can be integrated at any time, and the process can be done with no downtime.

Figure 7: That's better: The setup can be scaled seamlessly in width at all levels.

Conclusions

With the S3 plugin for Dovecot, Sirainen has expanded the scalability of mail at the storage level. Thus, a useful alternative is now available to previous solutions, such as sharding, whose inflexibility often causes problems in today's agile IT structures. The license fees the author charges for the plugin are undoubtedly a drawback. With a sensible setup, however, the costs can be recouped quickly because they remove the need for an expensive SAN on the storage side; instead, Ceph runs on off-the-shelf hardware. In terms of performance, Ceph – except the single-serial-write scenario – usually proves to be the superior solution and certainly has no reason to fear direct comparison with commercial storage solutions.

Those who operate an email setup and regularly face challenges when creating more space for user data should certainly take a closer look at the combination of Dovecot, the S3 plugin, and Ceph with Object Gateway. Instructions [7] explain how to set up Ceph with Object Gateway so that it cannot be distinguished from "real S3" clients.

Infos

  1. Ceph: http://ceph.com
  2. "Dream Team" by Martin Loschwitz, Linux Magazine , issue 165, pg. 20: http://www.linux-magazine.com/Issues/2014/165/Storage-Cluster
  3. Ceph Enterprise from Inktank: http://www.inktank.com/enterprise/
  4. Red Hat Storage Server: http://www.redhat.com/en/technologies/storage/storage-server
  5. Dovecot: http://www.dovecot.fi
  6. Documentation for Ceph Object Gateway: http://ceph.com/docs/master/radosgw/
  7. S3 trick for the Ceph Object Gateway: http://www.hastexo.com/resources/hints-and-kinks/configuring-radosgw-behave-amazon-s3

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Getting Ready for the New Ceph Object Store

    The Ceph object store remains a project in transition: The developers announced a new GUI, a new storage back end, and CephFS stability in the just released Ceph v10.2.x, Jewel.

  • Ceph object store innovations
    The Ceph object store remains a project in transition: The developers announced a new GUI, a new storage back end, and CephFS stability in the just released Ceph c10.2.x, Jewel.
  • Ceph and OpenStack Join Forces

    When building cloud environments, you need more than just a scalable infrastructure; you also need a high-performance storage component. We look at Ceph, a distributed object store and filesystem that pairs well in the cloud with OpenStack.

  • Comparing Ceph and GlusterFS
    Many shared storage solutions are currently vying for users’ favor; however, Ceph and GlusterFS generate the most press. We compare the two competitors and reveal the strengths and weaknesses of each solution.
  • Troubleshooting and maintenance in Ceph
    We look into some everyday questions that administrators with Ceph clusters tend to ask: What do I do if a fire breaks out or I run out of space in the cluster?
comments powered by Disqus