Amazon Simple Queue Service

Amazon Simple Queue Service (Amazon SQS) offers reliable and scalable hosted queues for storing and retrieving messages. By using Amazon SQS, you can move data between distributed components of your applications that perform different tasks without losing messages or requiring each component to be always available.

Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. A queue is a temporary repository for messages that are waiting processing.

Using Amazon SQS, you can decouple the components of an application so they run independently, with Amazon SQS easing message management between components. Any component of a distributed application can store messages in a fail-safe queue. Messages can contain up to 256 KB of text in any format. Any component can later retrieve the messages programmatically using the Amazon SQS API.

The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing. This means the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer are only intermittently connected to the network.

Architectural Overview

There are three main actors in the overall system:

The components of your distributed system
Queues
Messages in the queues

In the following diagram, your system has several components that send messages to the queue and receive messages from the queue. The diagram shows that a single queue, which has its messages (labeled A-E), is redundantly saved across multiple Amazon SQS servers.

Create Queue

Default Visibility Timeout

The default visibility timeout is a message is hidden after a message is retrieved from the queue.

Default – 30 seconds

Message Retention Period

How long the message will be present in our queue.

Default – 4 days

Maximum Message Size

The maximum message size is 256KB. Amazon SQS uses batch processing to transfer the message into the SQS.

Delivery Delay

The delivery delay is a message is hidden when it is first added to the queue.

Default – 0 seconds

Receive Message Wait Time

SQS is the reduction of the number of empty responses, when there are no messages available to return, in reply to a ReceiveMessage request sent to an Amazon SQS queue. Long polling allows the Amazon SQS service to wait until a message is available in the queue before sending a response. So unless the connection times out, the response to the ReceiveMessage request will contain at least one of the available messages (if any) and up to the maximum number requested in the ReceiveMessage call.

Message Attributes

Message attributes allows you to provide structured metadata items (such as timestamps, geospatial data, signatures, and identifiers) about the message. Message attributes are optional and separate from, but sent along with, the message body. This information can be used by the receiver of the message to help decide how to handle the message without having to first process the message body.

Message Attribute Data Types

String – Strings are Unicode with UTF-8 binary encoding.
Number – Numbers are positive or negative integers or floating point numbers.
Binary – Binary type attributes can store any binary data, for example, compressed data, encrypted data, or images.

SQS Long Polling

One benefit of long polling with Amazon SQS is the reduction of the number of empty responses, when there are no messages available to return, in reply to a ReceiveMessage request sent to an Amazon SQS queue. Long polling allows the Amazon SQS service to wait until a message is available in the queue before sending a response. So unless the connection times out, the response to the ReceiveMessage request will contain at least one of the available messages (if any) and up to the maximum number requested in the ReceiveMessage call.

Another benefit is helping to eliminate false empty responses, where messages are available in the queue but are not included in the response. This happens when Amazon SQS uses short (standard) polling, the default behavior, where only a subset of the servers (based on a weighted random distribution) are queried to see if any messages are available to include in the response.

SQS Delay Queues

Delay queues allow you to postpone the delivery of new messages in a queue for a specific number of seconds. If you create a delay queue, any message that you send to that queue will be invisible to consumers for the duration of the delay period. You can use CreateQueue to create a delay queue by setting the DelaySeconds attribute to any value between 0 and 900 (15 minutes). You can also turn an existing queue into a delay queue by using SetQueueAttributes to set the queue's DelaySeconds attribute.

Delay queues are similar to visibility timeouts in that both features make messages unavailable to consumers for a specific period of time. The difference between delay queues and visibility timeouts is that for delay queues, a message is hidden when it is first added to the queue, whereas for visibility timeouts, a message is hidden only after a message is retrieved from the queue.

Message Lifecycle

Component 1 sends Message A to a queue, and the message is redundantly distributed across the SQS servers.
When Component 2 is ready to process a message, it retrieves messages from the queue, and Message A is returned. While Message A is being processed, it remains in the queue and is not returned to subsequent receive requests for the duration of the visibility timeout.
Component 2 deletes Message A from the queue to avoid the message being received and processed again once the visibility timeout expires.

Supported Data Types

Strings
Numbers (Integers and Floats)
Lists
Maps
Structs (which are just nested Maps)

The Common Differences between ActiveMQ, Amazon SQS

BenchMark

http://dataexcursions.wordpress.com/2011/12/15/amazon-sqs-vs-rabbitmq