Quick AWS SQS SNS

Posted on October 2, 2021
Tags: aws

1 Broad Summary

2 SQS

Max message size is 256 KB
Typically middlemans one-to-one, NOT FANOUT

2.1 Mechanism

  1. Producer Send Msg to SQS
  2. SQS Send Msg to Consumer
  3. Consumer initiate Polling 4a. Consumer Process Msg 4b. Consumer Delete Msg

Message Details:

ID                              Size      MD5 of message body
ABC-CDE-FE32-EG12-ZZZZZZZZZZ    9 bytes   4901d08f14cacf11650dadaac7e22067

Sender account ID     Sent                      First received
ABCDEFFFFFFFFFFFFFFF  7/8/2022, 12:26:26 EDT    7/8/2022, 12:26:50 EDT

Receive count   Message attributes count  Message attributes size   MD5 of message attributes
2               -                         -                         -
  • Note that it is possible to poll for message but not delete it. This will make the *Receive count” go up.

2.2 Common pattern

  • A very common pattern is throwing SQS infront of a lambda or db or microservice.(really just throw it in front of anything important)
    • Retrys if fail
    • If message processing lambda needs to be taken down for service, means messages can be left in queue to be later processed.
    • Loosen tight dependencies by giving an agnostic message queue between services

2.3 Examples

2.3.1 Ex1

import json
import boto3 

def lambda_handler(event, context):
    # TODO implement
    sqs = boto3.client('sqs')
    sqs.send_message(
        QueueUrl = "https://sqs.us-east-1.amazonaws.com/637189393063/myqueue",
        MessageBody = "hi"
        )
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

3 Counter example: SQS vs SNS

3.1 Usecase

REMEMBER: How you throw SQS in-front of anything important?

  • SQS-SNS fanout pattern
    • SNS is typically used in conjunction with SQS
    • SQS fans out to multiple SNS queues infront of individual microservices
      • This way when one microservice goes down, data isn’t lost and instead kept in queue