AWS SQS Explained: A Complete Beginner-to-Advanced Guide (With Examples)
Amazon Simple Queue Service (SQS) is one of the most powerful yet simplest AWS services for building distributed, decoupled, and scalable applications. Whether you are designing microservices, event-driven systems, or background job processingβSQS is the backbone that ensures reliability and smooth message flow.
In this blog, we will explore:
- What AWS SQS is
- Why SQS is needed
- Types of queues
- Architecture
- Features
- Real-world use cases
- Dead-letter queues
- Example code (Node.js +php+laravel)
Best practices
π What is AWS SQS?
AWS SQS (Simple Queue Service) is a fully managed message queue that enables microservices and distributed systems to communicate asynchronously.
π You send a message β it stays in a queue β a worker/consumer processes it.
You donβt worry about:
- Server maintenance
- Scaling
- Failures
- Message loss
AWS handles everything automatically.
β¨ Why Do We Need SQS?
Modern systems need to process tasks asynchronously to stay fast and efficient.
β Without SQS (Direct communication)
- Services are tightly coupled
- If one service fails, the entire application may fail
- Traffic spikes can overload systems
- Long-running tasks slow down UI responses
β With AWS SQS
- Components are loosely coupled
- No message is lost
- Traffic spikes are handled smoothly
- Systems remain scalable and fault-tolerant
- Workers process tasks in the background
π¦ Types of SQS Queues
1. Standard Queue (High Throughput)
- Nearly unlimited messages per second
- Best-effort ordering (not guaranteed)
- Messages delivered at least once (duplicate possible)
Use Case: sending emails, logs, notifications, processing jobs
2. FIFO Queue (First-In-First-Out)
- Guaranteed order
- No duplicates
- Limited throughput (3,000 msgs/s)
Use Case: payment processing, order events, inventory updates
βοΈ AWS SQS Architecture (Simple Diagram)
Producer ---> SQS Queue ---> Consumer (Worker/App)Add DLQ (Dead Letter Queue) for failed messages:
Producer ---> Main Queue ---> Consumer
|
v
DLQ (Failed Messages)π Important SQS Features
β Message Retention (1 min β 14 days)
β Visibility Timeout
Prevents other workers from processing the same message.
Default: 30 seconds
Example: Worker is processing β message becomes invisible β if worker fails β message reappears.
β Dead Letter Queue (DLQ)
Failed messages are moved after max retries.
β Long Polling (recommended)
Reduces cost by waiting until a message is available.
β Encryption (SSE)
Data secured using AWS KMS.
β Message Batching
Send/receive up to 10 messages at once.
π§© Real-World Use Cases
| Use Case | Description |
|---|---|
| Send OTP/Emails | App queues sending requests, worker sends mail |
| Process payments | FIFO queue ensures order |
| Video processing | Upload β Queue β Worker compresses video |
| Notification systems | Multiple microservices send events |
| IoT devices | Billions of messages handled easily |
| Order management | Highly reliable message flow |
π΅ EXAMPLE 1 β Basic Email Queue With SQS (Node.js)
A simple app sends an email request to SQS, worker processes it later.
Architecture
Frontend β API β SQS Queue β Worker β Send Email1οΈβ£ Step 1: Install AWS SDK
npm install @aws-sdk/client-sqs nodemailer2οΈβ£ Step 2: Send Message (Producer)
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
const client = new SQSClient({ region: "ap-south-1" });
export const queueEmail = async () => {
const params = {
QueueUrl: process.env.SQS_URL,
MessageBody: JSON.stringify({
to: "user@example.com",
subject: "Welcome to our platform!",
timestamp: Date.now(),
}),
};
await client.send(new SendMessageCommand(params));
console.log("Message added to queue!");
};3οΈβ£ Step 3: Worker to Receive & Process Message
import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from "@aws-sdk/client-sqs";
import nodemailer from "nodemailer";
const sqs = new SQSClient({ region: "ap-south-1" });
async function processQueue() {
while (true) {
const data = await sqs.send(
new ReceiveMessageCommand({
QueueUrl: process.env.SQS_URL,
MaxNumberOfMessages: 1,
WaitTimeSeconds: 20, // Long polling
})
);
if (!data.Messages) continue;
for (const msg of data.Messages) {
const payload = JSON.parse(msg.Body);
console.log("Processing email:", payload.to);
await sendEmail(payload);
await sqs.send(
new DeleteMessageCommand({
QueueUrl: process.env.SQS_URL,
ReceiptHandle: msg.ReceiptHandle,
})
);
}
}
}
async function sendEmail(data) {
console.log("Email sent to:", data.to);
}
processQueue();
π΅ EXAMPLE 2 β Order Processing With FIFO Queue (Node.js)
Ensures correct order of order events.
π FIFO Rules
- Queue name must end with
.fifo - Requires
MessageGroupId - Supports exactly-once processing
Architecture
Order Service β orders.fifo β Worker β Update Inventory β Update DBSend Order Message (FIFO)
await client.send(
new SendMessageCommand({
QueueUrl: process.env.FIFO_SQS_URL,
MessageBody: JSON.stringify({
orderId: "ORD_10021",
action: "ORDER_PLACED",
}),
MessageGroupId: "order-events",
MessageDeduplicationId: Number(Date.now()).toString(),
})
);
Why?
- MessageGroupId = all messages in FIFO group ordered
- DeduplicationId = avoids duplicates
Receive & Process FIFO Messages
const data = await client.send(
new ReceiveMessageCommand({
QueueUrl: process.env.FIFO_SQS_URL,
MaxNumberOfMessages: 1,
WaitTimeSeconds: 20,
})
);
FIFO processes one group at a time, ensuring order.
π΅ EXAMPLE 3 β Full Video Processing Workflow (Laravel)
This is a real-world heavy-use scenario.
Architecture
User Uploads Video β API β SQS Queue β Worker β Compress β Store β Notify UserSend Video Job to SQS (Laravel)
$sqs = new SqsClient([
'region' => env('AWS_DEFAULT_REGION'),
'version' => 'latest',
]);
$sqs->sendMessage([
'QueueUrl' => env('SQS_VIDEO_QUEUE'),
'MessageBody' => json_encode([
'user_id' => $user->id,
'file_path' => $uploadedFile,
]),
]);Worker Consuming SQS (Laravel)
public function handle()
{
$client = new SqsClient([
'region' => env('AWS_DEFAULT_REGION'),
'version' => 'latest',
]);
while (true) {
$result = $client->receiveMessage([
'QueueUrl' => env('SQS_VIDEO_QUEUE'),
'MaxNumberOfMessages' => 1,
'WaitTimeSeconds' => 20,
]);
if (!empty($result->get('Messages'))) {
foreach ($result->get('Messages') as $message) {
$data = json_decode($message['Body'], true);
$this->compressVideo($data['file_path']);
$client->deleteMessage([
'QueueUrl' => env('SQS_VIDEO_QUEUE'),
'ReceiptHandle' => $message['ReceiptHandle'],
]);
}
}
}
}π΅ EXAMPLE 4 β AWS Lambda Trigger With SQS
Zero servers. Just a Lambda that self-scales.
Configure:
SQS Queue β Lambda Trigger β Process β Auto-delete
Lambda Example (Node.js)
exports.handler = async (event) => {
for (const record of event.Records) {
const message = JSON.parse(record.body);
console.log("Processing:", message);
}
};SQS pushes messages in batches to Lambda.
π΅ EXAMPLE 5 β Dead Letter Queue (DLQ) in Real Use
Scenario:
- Worker fails 5 times
- Message moves automatically to DLQ
Create Redrive Policy
maxReceiveCount = 5
deadLetterTargetArn = arn:aws:sqs: region:account:myDLQWorker Fails Example
try {
throw new Error("Database Timeout");
} catch (err) {
console.log("Failed:", err.message);
// SQS will retry automatically until maxReceiveCount
}Once retries exceed 5 β SQS moves it to DLQ.
π΅ EXAMPLE 6 β Batch Processing (10 Messages at Once)
Improves cost & performance.
Send Batch (Node.js)
await client.send(
new SendMessageBatchCommand({
QueueUrl: process.env.SQS_URL,
Entries: [
{
Id: "1",
MessageBody: "Task #1"
},
{
Id: "2",
MessageBody: "Task #2"
}
]
})
);Receive Batch
const data = await sqs.send(
new ReceiveMessageCommand({
QueueUrl: process.env.SQS_URL,
MaxNumberOfMessages: 10,
WaitTimeSeconds: 20,
})
);π΅ EXAMPLE 7 β Visibility Timeout Failure Scenario
Step-by-step:
- Worker receives message
- Visibility timeout = 30 sec
- Worker takes 90 sec β FAIL
- Message reappears β another worker picks it
- Duplicate processing occurs
Fix: Extend Visibility Timeout
await sqs.send(
new ChangeMessageVisibilityCommand({
QueueUrl: process.env.SQS_URL,
ReceiptHandle: msg.ReceiptHandle,
VisibilityTimeout: 120, // increase time
})
);Best Practices
β Enable long polling
β Use DLQ for all queues
β Use FIFO queues for ordered tasks
β Set correct visibility timeout
β Monitor SQS using CloudWatch
β Batch messages to reduce cost
β Encrypt messages with KMS
Conclusion
AWS SQS is a highly scalable, fault-tolerant, distributed queuing system that allows modern apps to run smoothly without losing messages. Whether building microservices, background jobs, or large asynchronous systemsβSQS is a must-use AWS service.
