Only use Amazon SQS if you need a high-latency high-concurrency service

(written by lawrence krubner, however indented passages are often quotes). You can contact lawrence at:, or follow me on Twitter.

Also interesting, don’t use Amazon SQS unless your needs fit this very specific model where you can deal with the latency and make up for it by being highly concurrent.

I just did some benchmarking the other day to compare Amazon SQS with RabbitMQ. Publishing and consuming 10,000 messages serially in 2 threads (one publishing the other consuming) on an EC2 instance took over 6 minutes using SQS and 12 seconds using RabbitMQ. I didn’t test ActiveMQ since their clustering is pretty weak, so we threw them out before getting to the testing stage. I would stay away from SQS unless your volume needs are pretty low.

This response was very interesting:

Just a note on SQS, it is a high-latency high-concurrency service. You can scale SQS to very high throughput with event-driven frameworks like eventmachine or twisted. Each request will take .25 seconds, but if you have 100 requests in flight at once you could do 400 requests per second, and it scales out further than that in my experience.

That said, RabbitMQ is still a better choice for high throughput flows and Storm in particular. The main advantages of SQS are its universal availability (anything with internet access can talk to it) and high availability.

Post external references

  1. 1!topic/storm-user/rd3JD1p0wns