Fanout is the pattern where one event needs to reach several independent consumers. An order is placed, and the warehouse needs to pick it, the email service needs to send a receipt, the fraud check needs to score it and analytics wants a copy. The order service shouldn’t know about any of them. With Amazon SNS it publishes once to a topic, and SNS delivers a copy to each subscription.
That much is easy to say. What interviewers probe is what “independent” means in practice: what happens when one consumer is down, slow or broken, and who is responsible for retrying.
What Publish actually promises
When Publish returns a MessageId, SNS has accepted and stored the message. That’s the whole promise. It doesn’t mean the email was sent or the warehouse got it. Delivery to each subscription happens afterwards, asynchronously, and the publisher never hears how it went.
This has two consequences people miss. First, the publisher can’t use the response to decide whether the business process succeeded; if it needs that, it needs a different design (a synchronous call, or a saga with replies). Second, if the publisher writes to its database and then publishes, a crash between the two loses the event. The usual fix is an outbox: write the event to a table in the same transaction as the order, and publish from that table. That’s a database problem, not an SNS one, but it’s the first follow-up question in many interviews.
Every subscription stands alone
SNS treats each subscription as its own delivery. If the warehouse queue accepts the message and the HTTPS webhook for a partner returns 503, the warehouse isn’t affected and the publisher isn’t told. SNS retries the webhook according to its delivery policy, which for HTTP/S endpoints you can tune (number of retries, backoff, throttle). For AWS-managed endpoints such as SQS and Lambda, SNS retries for a long time on its own, because those failures are usually transient.
When retries are exhausted, the message is dropped for that subscription unless you’ve attached a subscription-level dead-letter queue (an SQS queue set in the subscription’s redrive policy). Note the level: the DLQ belongs to the subscription, not the topic, so one broken subscriber can have its failures parked without touching the others.
The other independence that matters is pace. SNS pushes. If you subscribe a Lambda function directly and publish 5,000 messages in a second, SNS invokes the function as fast as it can, and the function scales up (or gets throttled) accordingly. A Lambda function that writes to a small RDS database can take that database down. That’s the main reason for the next pattern.
SNS to SQS: the queue per consumer pattern
The common production shape is SNS fanning out to one SQS queue per consumer, with each consumer reading from its own queue:
┌──> SQS warehouse-queue ──> warehouse workers
order-service ──> SNS order-events
├──> SQS email-queue ─────> email Lambda
└──> SQS fraud-queue ─────> fraud workersThe queue gives each consumer things SNS alone doesn’t:
- Buffering. If the email service is deployed or down for 20 minutes, its messages wait in
email-queue(up to the queue’s retention period, 4 days by default). Nothing is lost and SNS isn’t retrying anything, because the queue accepted the message right away. - Its own pace. The fraud workers can poll 10 at a time while the warehouse uses batch processing. A spike becomes a backlog rather than a flood.
- Its own retries and DLQ. Each queue has its own visibility timeout and redrive policy, sized for that consumer.
- Replay by redrive. When a bug is fixed, you redrive the consumer’s DLQ without republishing to everyone.
The thing that trips people up when they set this up is permissions. The queue needs a resource policy that lets SNS send to it, scoped to your topic. Here’s a policy for email-queue (account IDs and names are illustrative):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "sns.amazonaws.com" },
"Action": "sqs:SendMessage",
"Resource": "arn:aws:sqs:eu-west-1:111122223333:email-queue",
"Condition": {
"ArnEquals": { "aws:SourceArn": "arn:aws:sns:eu-west-1:111122223333:order-events" }
}
}
]
}Without the aws:SourceArn condition, any SNS topic in any account could send to the queue. Without the statement at all, the subscription exists, Publish succeeds, and the queue stays empty, which looks exactly like “SNS is broken” until you check the topic’s NumberOfNotificationsFailed metric. If the queue is encrypted with a customer managed KMS key, the key policy also has to let SNS use it.
You’ll also usually turn on raw message delivery for SQS subscriptions. Without it, each SQS message body is an SNS JSON envelope (Type, MessageId, TopicArn, Message, signature fields), and your consumer has to unwrap Message. With it, the body is exactly what was published.
Filter policies: subscribers that only want some events
The fraud service doesn’t need a £12 order, and the EU warehouse doesn’t need US orders. Instead of filtering in code (paying to receive and discard), you attach a filter policy to the subscription. SNS evaluates it against message attributes (or the body, if you set the filter policy scope to MessageBody) and only delivers matches.
The matching rules are worth knowing exactly: keys are combined with AND, values listed under one key are OR, and if the policy names an attribute the message doesn’t have, the message isn’t delivered. Here’s a small model of that subset in Python. It’s not the SNS engine (which supports prefixes, exists, IP ranges and more), just the three rules:
# A tiny subset of SNS filter-policy matching: exact strings, anything-but, and numeric ranges.
def matches(policy, attrs):
for key, rules in policy.items():
if key not in attrs:
return False
value = attrs[key]
ok = False
for rule in rules:
if isinstance(rule, str):
ok |= value == rule
elif "anything-but" in rule:
ok |= value not in rule["anything-but"]
elif "numeric" in rule:
op, n = rule["numeric"]
ok |= isinstance(value, (int, float)) and {">=": value >= n, "<": value < n}[op]
if not ok:
return False # every key must match (AND); values inside a key are OR
return True
subscriptions = {
"warehouse-queue": {"country": ["DE", "FR", "NL"]},
"fraud-queue": {"total": [{"numeric": [">=", 1000]}]},
"analytics": {}, # no filter policy: receives everything
}
messages = [
{"orderId": "ord_1", "country": "DE", "total": 49.5},
{"orderId": "ord_2", "country": "US", "total": 2300},
{"orderId": "ord_3", "total": 80}, # missing country attribute
]
for m in messages:
got = [s for s, p in subscriptions.items() if matches(p, m)]
print(m["orderId"], "->", ", ".join(got))Output from python3:
ord_1 -> warehouse-queue, analytics
ord_2 -> fraud-queue, analytics
ord_3 -> analyticsord_3 is the interesting row. The publisher forgot the country attribute, so the warehouse never sees the order, silently. Filtered-out messages aren’t failures and don’t go to a DLQ. If you rely on filters for business routing, validate attributes at the publisher and watch the NumberOfNotificationsFilteredOut metrics for surprises.
Ordering and duplicates
Standard SNS topics don’t guarantee order and can deliver a message more than once. Your consumers need to be idempotent anyway (SQS behind them is at-least-once too), so the practical rule is: carry an event ID and a version or timestamp, and let consumers ignore duplicates and stale updates.
If you truly need ordering, SNS FIFO topics preserve order per message group and deduplicate publishes, and they deliver to SQS queues. They trade throughput for that guarantee, and the ordering only holds end to end if the consumer side preserves it too.
Interview practice: the partner webhook that slows everyone down
A team subscribes three things directly to an order-events topic: a Lambda that writes to Postgres, an HTTPS webhook run by a partner, and an email address for the ops team. During a sale, Postgres runs out of connections and the partner complains about missing events. How would you redesign this?
Answer
The symptoms come from SNS pushing directly to consumers that can’t absorb a spike. The Lambda scales out with the publish rate, and each concurrent execution opens a database connection, so Postgres hits its limit. The partner webhook is probably returning errors under load; once SNS’s HTTP delivery retries are exhausted, those messages are dropped because there’s no subscription DLQ.
I’d put an SQS queue behind the Lambda and set the event source mapping’s maximum concurrency so it can’t open more connections than the database allows. The backlog drains at a safe rate. For the partner, I’d attach a subscription DLQ so failed deliveries are kept, tune the delivery policy’s backoff, or better, have our own worker read from a queue and call the partner with retries and an idempotency key we control. The email subscription is fine for low-volume alerts but shouldn’t receive every order; I’d add a filter policy or move it to a separate alerts topic.
Independence is the point I’d say out loud: each consumer gets its own queue, concurrency, retries and DLQ, so the partner’s outage can’t affect the database and vice versa.
Keep going
- AWS interview questions
- AWS MCQs
- AWS SQS delivery and visibility timeouts
- AWS EventBridge scheduling
- Fanout to Amazon SQS queues (AWS docs)