July 21st, 2016
(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at: firstname.lastname@example.org
I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is pretty weak, so we threw them out before getting to the testing stage. I would stay away from SQS unless your volume needs are pretty low.
This response was very interesting:
Just a note on SQS, it is a high-latency high-concurrency service. You can scale SQS to very high throughput with event-driven frameworks like eventmachine or twisted. Each request will take .25 seconds, but if you have 100 requests in flight at once you could do 400 requests per second, and it scales out further than that in my experience.
That said, RabbitMQ is still a better choice for high throughput flows and Storm in particular. The main advantages of SQS are its universal availability (anything with internet access can talk to it) and high availability.