Running ActiveMQ on AWS Fargate

Introduction

There is not much to be said for ActiveMQ - when it comes to message brokers, it’s one of the most well-known and trusted ones. So, it’s not a surprise that AWS provides its own flavor: Amazon MQ. While Amazon MQ supports both ActiveMQ and RabbitMQ, I’ll focus on ActiveMQ throughout this post¹.

When operating SaaS infrastructure, any chance to use a managed service is an opportunity to better utilize the most scarce resource: software engineers. If you can save even a few hours per month for your team so they focus on core products instead of operating middleware components, it is a win. Of course, such wins usually come with a cost that needs to be justified. And this is what this post is about!

About Amazon MQ costs

Choosing a managed service such as Amazon MQ could be the best option during the early stages. It allows you to:

• Focus on core product development instead of deploying and operating middleware
• Reduce time to market

But when your growth leads Amazon MQ to the Top 5 of your AWS bill, you start to see things differently. Let’s do some comparisons²:

Instance Type	Amazon MQ	EC2	Amazon MQ vs EC2
t3.micro	$0.02964 /hr	$0.0114 /hr	+160%
m5.large	$0.32100 /hr	$0.1070 /hr	+200%

This is not something you can ignore.

Actually, it is even worse: considering that Amazon MQ does not support reserved instances, nor is it covered by Savings Plans, if one chooses to commit to the compute costs, the difference is even greater since EC2 costs would be even less.

So, there is an obvious choice: abandon Amazon MQ and switch to EC2. But then you would have to manage those EC2 servers running ActiveMQ…

There is a serverless alternative!

Let’s review the requirements mentioned earlier while adding some that were implied:

• We need a serverless solution so we don’t end up managing middleware components such as ActiveMQ
• We need a cost-efficient solution
• We need a high-availability solution (similar to what Amazon MQ offers - i.e. support for failover)
• We need security features (similar to what Amazon MQ offers - i.e. support for encrypted communication)

Starting with our first requirement, a serverless solution, my first idea was ECS with Fargate. But what about the rest of the requirements?

Cost efficiency

ECS costs can be reduced when Savings Plans are used, contrary to Amazon MQ. Here’s the cost comparison without Savings Plans²:

Infrastructure	Amazon MQ	ECS	Amazon MQ vs EC2
t3.micro (2 vCPU / 1GiB)	$0.02964 /hr	$0,072594 /hr	-59%
m5.large (2 vCPU / 8GiB)	$0.32100 /hr	$0,099042 /hr	+224%

Interesting notes:

• Even before Saving Plans, ECS/Fargate is a better option for production-grade instances
• We are comparing apples with apples: Amazon MQ comes with no long-term commitments so it’s only fair to compare with the on-demand costs of ECS
• For low-end instances, Amazon MQ is cheaper. This is because ECS cost is CPU driven and both t3.micro and m5.large instances have 2 vCPUs. Of course, this is not a fair comparison (t3 is burstable, Fargate is not) but it brings up another advantage on the ECS side: You can configure the available resources (CPU/Memory) with far more granular options than the limited list of Amazon MQ instance types. An ECS task with 0.5 vCPU and 4GiB of memory would be ok for most UAT/Staging environments and cheaper than an mq.t3.micro.

High-availability with EFS

We wouldn’t switch from Amazon MQ to ECS unless the new solution meets the high-availabilty requirements. This was actually very easy to implement by using the same storage with Amazon MQ when it comes to high availability and message durability: Amazon Elastic File System (EFS).

By configuring ActiveMQ to use a specific directory for KahaDB and dataDirectory, and then mounting an EFS file system to that directory (ACTIVEMQ_DATA), all data will be persisted on EFS. Should a broker fail, the replacement would recover data (and transactions) from EFS once the failover is complete.

Security

Security at-rest is an easy task, considering that EFS can be encrypted.

When it comes to security in-transit, there is more to be done. Amazon MQ uses its own certificates so TLS connections are straightforward. If we need to provide the same security features, we need to issue (or buy) our own certificate and then configure each client to trust the certificate or the issuing CA. I plan to write a brief guide on how to do that using OpenSSL, but until then, self-signed certificates are good enough. We then add the server key in the container image and configure Jetty to use it.

Step by step

• Issue server certificates (you can use this reference script (link to script on GH))
• Build the docker image and publish it on ECR
• Create the infrastructure (ECS Cluster / Service / Tasks, EFS) (link to Terraform on GH)

---

Photo by Florencia Viadana on Unsplash

• AWS
• ActiveMQ
• ECS