I built an event-driven architecture on AWS to familiarize myself with core services and how they integrate to form scalable, serverless applications. The system simulated an order processing pipeline with notifications.

Architecture Overview

The workflow was:

  • Amazon API Gateway received HTTP POST requests from a client
  • Amazon SQS acted as a message queue for reliability
  • A Lambda function consumed SQS messages, parsed order details, and wrote them into a DynamoDB table
  • DynamoDB Streams captured table changes
  • A second Lambda function processed new records from the stream and published notifications to Amazon SNS
  • Amazon SNS fanned out messages to subscribed external systems (e.g., email, HTTP endpoints)

AWS Event Driven ArchitectureEvent-driven architecture built with API Gateway, SQS, Lambda, DynamoDB, Streams, and SNS using Cacoo.

IAM Policies and Roles

I created custom IAM policies following least-privilege principles:

  • Lambda-Read-SQS: DeleteMessage, ReceiveMessage
  • Lambda-Write-DynamoDB: PutItem, DescribeTable
  • Lambda-DynamoDBStreams-Read: GetRecords, DescribeStream
  • Lambda-SNS-Publish: Publish, GetTopicAttributes

Each Lambda function used its own execution role with only the permissions required.

Lambda Implementations

Lambda 1 (SQS → DynamoDB)

  • Triggered by SQS
  • Parsed incoming payloads and inserted structured records into the Orders DynamoDB table using Python and Boto3

Lambda 2 (DynamoDB Streams → SNS)

  • Triggered by DynamoDB Streams (new item images)
  • Published order data as SNS messages for subscribers

Testing & Verification

I tested the pipeline end-to-end using API Gateway and verified each component:

  • Sent POST requests through API Gateway with JSON payloads
  • Verified message flow through SQS, DynamoDB, Streams, and SNS
  • Confirmed delivery by receiving email notifications from SNS

Takeaways

This project allowed me to build confidence with AWS services and understand how to connect them in a serverless pipeline. I learned how to:

  • Connect API Gateway, SQS, Lambda, DynamoDB, and SNS into a working event-driven system
  • Design secure IAM policies and roles for serverless applications
  • Debug distributed workflows with CloudWatch logs and step-by-step validation
  • Apply event-driven principles to real-world cloud applications