“Does Lambda retry?” has three different answers, and the interview question is usually checking whether you know which one applies. It depends on how the function was invoked. Get that wrong and you either lose events you thought were retried, or you send customers two receipts.
Three invocation models, three retry owners
Synchronous invocations are the ones where the caller waits for the result: API Gateway, an Application Load Balancer, or your code calling Invoke with RequestResponse. If the function throws or times out, Lambda returns the error to the caller and stops there. Lambda itself does not retry. The AWS SDK might retry certain errors (throttling, for example) depending on its configuration, and API Gateway won’t retry at all. A user who sees a 502 and clicks again is the retry.
Asynchronous invocations are the fire-and-forget kind: S3 event notifications, SNS, EventBridge, or Invoke with InvocationType=Event. Lambda puts the event in an internal queue and returns 202 to the caller immediately. If the function errors, Lambda retries it twice more by default, waiting about a minute before the first retry and about two before the second. You can set MaximumRetryAttempts to 0, 1 or 2, and MaximumEventAgeInSeconds up to six hours. Throttling and service errors are retried for longer, until the event is too old. When retries run out, the event goes to an on-failure destination or a DLQ if you configured one; otherwise it’s discarded.
Event source mappings poll a source for you: SQS, Kinesis, DynamoDB Streams, Kafka. Here retry behaviour is the source’s:
- With SQS, a failed batch isn’t deleted, so its messages reappear after the queue’s visibility timeout and are retried until the queue’s redrive policy sends them to a DLQ. By default one bad message fails the whole batch. Turn on
ReportBatchItemFailuresand return the IDs of only the failed messages, so the good ones are deleted. - With Kinesis and DynamoDB Streams, records are processed in order per shard, and by default Lambda retries a failing batch until the records expire, which blocks that shard. You bound this with
MaximumRetryAttempts,MaximumRecordAgeInSeconds,BisectBatchOnFunctionError(split the batch to isolate the bad record) and an on-failure destination.
The shared lesson: a handler can be invoked more than once for the same event. Async and event source deliveries are at-least-once even when nothing failed, and every retry path above re-sends the same payload.
The timeout that happens after the work
Retries feel safe if you picture them as “the function failed, so nothing happened, so run it again.” That’s often false. Consider a handler that sends a receipt email when an OrderPlaced event arrives via EventBridge. The email provider is slow, the call succeeds at 5.9 seconds, and the function’s 6-second timeout fires before the handler returns. Lambda records Task timed out after 6.00 seconds, treats the invocation as failed, and retries. The customer gets two receipts.
The same thing happens when a handler writes to a database and then crashes on a later line, or when the function finishes but the response is lost. From Lambda’s side, “failed” means “didn’t report success”, not “had no effect”.
Making a handler idempotent
The standard fix is an idempotency record. Before doing the side effect, the handler claims a key derived from the event (the order ID, or the event’s own ID) with a conditional write. If the key is already marked done, it returns without repeating the work.
Here’s a runnable simulation in Node. The store mimics a DynamoDB conditional put by throwing ConditionalCheckFailedException when the key exists. invokeAsync mimics Lambda’s async retry loop. The email call is rigged so the first attempt sends the email and then times out:
// Simulates Lambda's async retry of one event against two handlers.
// The "store" mimics a DynamoDB conditional put: it throws if the key already exists.
const store = new Map();
function putIfAbsent(key, value) {
if (store.has(key)) { const e = new Error('The conditional request failed'); e.name = 'ConditionalCheckFailedException'; throw e; }
store.set(key, value);
}
let emailsSent = 0;
let flaky = true; // the first attempt times out after sending
async function sendReceipt(orderId) {
emailsSent++;
if (flaky) { flaky = false; throw new Error('Task timed out after 6.00 seconds'); }
}
const naive = async (event) => { await sendReceipt(event.orderId); };
const idempotent = async (event) => {
const key = `receipt#${event.orderId}`;
try { putIfAbsent(key, { status: 'IN_PROGRESS' }); }
catch (e) {
if (e.name !== 'ConditionalCheckFailedException') throw e;
if (store.get(key).status === 'DONE') return 'duplicate, skipped';
// IN_PROGRESS from a failed attempt: fall through and try again
}
await sendReceipt(event.orderId);
store.set(key, { status: 'DONE' });
return 'sent';
};
async function invokeAsync(handler, event, maxRetries = 2) {
for (let attempt = 1; attempt <= maxRetries + 1; attempt++) {
try { const r = await handler(event); console.log(` attempt ${attempt}: ok ${r ?? ''}`); return; }
catch (e) { console.log(` attempt ${attempt}: ${e.message}`); }
}
console.log(' retries exhausted, event goes to the on-failure destination');
}
const event = { orderId: 'ord_7731' };
console.log('naive handler'); await invokeAsync(naive, event);
console.log(` emails sent: ${emailsSent}`);
emailsSent = 0; flaky = true; store.clear();
console.log('idempotent handler'); await invokeAsync(idempotent, event);
console.log(' duplicate delivery of the same event');
await invokeAsync(idempotent, event);
console.log(` emails sent: ${emailsSent}`);Output from node retries.mjs:
naive handler
attempt 1: Task timed out after 6.00 seconds
attempt 2: ok
emails sent: 2
idempotent handler
attempt 1: Task timed out after 6.00 seconds
attempt 2: ok sent
duplicate delivery of the same event
attempt 1: ok duplicate, skipped
emails sent: 2Read the second block carefully, because it’s the honest version of idempotency. The record did its job for the later duplicate delivery: that one was skipped. But the retry after the timeout still sent a second email, because the first attempt’s side effect happened after it claimed the key and before it could mark it done. The record can’t know whether the email went out.
That gap is closed by pushing the same key into the side effect. Most payment and email APIs accept an idempotency key; pass receipt#ord_7731, and the provider dedupes the second send. For your own database, make the write itself conditional (attribute_not_exists(orderId)) or state-setting, so repeating it changes nothing. The idempotency record handles “this event arrived twice”; the keyed side effect handles “this attempt partly ran”.
In real code, the record is a DynamoDB item written with ConditionExpression: 'attribute_not_exists(pk)' and a TTL so old keys expire. You don’t have to build it yourself: Powertools for AWS Lambda ships an idempotency utility that does the claim, stores the result and handles the in-progress case with an expiry, so a crashed attempt doesn’t block retries forever.
Gotcha
Pick the key from the business event, not from the invocation. The
aws_request_idin the Lambda context is different on each retry, so using it as the idempotency key dedupes nothing.
Configuring the async path deliberately
Defaults are rarely right for every function. A sensible configuration for the receipt function looks like this (illustrative CLI; nothing here was run against AWS):
aws lambda put-function-event-invoke-config \
--function-name send-receipt \
--maximum-retry-attempts 2 \
--maximum-event-age-in-seconds 3600 \
--destination-config '{"OnFailure":{"Destination":"arn:aws:sqs:eu-west-1:111122223333:receipt-failures"}}'An hour is enough to ride out a provider blip; a receipt that arrives six hours late is worse than an alert. The on-failure destination records the event plus the error details, which a plain DLQ doesn’t (a DLQ gets only the original event). Either way, alarm on it, or exhausted retries vanish quietly.
Also think about the timeout itself. Set it from measured latency of the slowest dependency plus margin, and give downstream SDK calls shorter timeouts than the function’s, so a slow call fails inside your code where you can handle it, rather than the runtime killing the invocation mid-write.
Interview practice: retrying a stream that’s stuck
A Lambda function consumes a DynamoDB stream and updates a search index. One malformed record makes the function throw. The team notices that search hasn’t updated for two hours for some items, while others are fine. Explain why and how you’d fix it.
Answer
With a stream event source, Lambda processes each shard in order and, by default, keeps retrying a failing batch until the records expire from the stream (24 hours for DynamoDB Streams). The malformed record makes the batch fail every time, so that shard is stuck behind it: nothing after it is processed. Items on other shards keep flowing, which is why only some items are stale.
I’d fix the configuration first: set BisectBatchOnFunctionError so Lambda splits the batch and isolates the bad record, set a bounded MaximumRetryAttempts and MaximumRecordAgeInSeconds, and add an on-failure destination so the skipped record is kept for inspection. Then return partial batch failures (ReportBatchItemFailures) so successfully processed records aren’t retried. Finally, because bisecting and retries will reprocess records, the index update has to be idempotent: upsert by item key with the stream record’s sequence number or a version, ignoring older versions. And I’d add an alarm on the mapping’s IteratorAge metric, since a growing iterator age is exactly what this incident looks like from the outside.