AWS Lambda: The Complete Guide — From Zero to Expert

AWS Lambda is one of the most widely used services in modern cloud and DevOps architectures — but many engineers still struggle to understand when to actually use it.

Should you use Lambda or EC2?
When does serverless make sense?
What are the real-world scenarios?

In this guide, we’ll go from zero to advanced — covering how Lambda works, when to use it, key configurations, and production-grade patterns like API Gateway integration and SAM templates.

By the end, you’ll not just understand Lambda — you’ll know how to use it in real systems.

Lambda vs EC2: When to Use What

EC2 gives you full control over virtual servers — OS, networking, storage, patching — while Lambda abstracts all of that away.​

When to Use Lambda

• Event-driven workloads: S3 file uploads triggering processing, DynamoDB stream handlers

• API backends: Lightweight REST/GraphQL APIs behind API Gateway

• Scheduled tasks: Cron-like jobs (e.g., daily tenant reports like the Daily-tenant-report function in the screenshot)

• Chatbot/IoT processing: Handling Alexa skills, IoT device data​

• Automation: Infrastructure tasks triggered by CloudTrail or Config rules​

When to Use EC2

• Long-running processes (>15 minutes)

• Stateful applications needing persistent memory

• Legacy monolithic apps requiring specific OS configurations

• GPU/specialized hardware workloads​

Hybrid Approach

Many organizations use both — Lambda for bursty, event-driven tasks and EC2 for steady-state workloads requiring fine-grained control.

Creating a Lambda Function (Step by Step)

Step by Step When you click Create function in the console, you see four options: ​

1. Author from Scratch Start with a Hello World example. You pick a runtime (e.g., nodejs24.x, python3.12), name your function, and Lambda sets up a basic handler.

   

2. Use a Blueprint Pre-built sample code for common use cases — S3 thumbnail generation, DynamoDB processing, Kinesis stream readers. Great for learning.

   

3. Container Image Deploy your function as a Docker container image stored in Amazon ECR. More on this in the advanced section below.

   

Architecture: arm64 vs x86_64

When creating a function, you choose the instruction set architecture:

Scenario: If you're writing a Python-based daily report generator (like Daily-tenant-report), arm64 is an easy win — most Python packages support it and you save money.

Lambda Configuration Deep Dive

General Configuration

• Memory: 128 MB to 10,240 MB. CPU scales proportionally with memory.

• Timeout: 1 second to 15 minutes max.

• Ephemeral storage (/tmp): 512 MB to 10,240 MB for temporary files.

Environment Variables

Key-value pairs injected at runtime. Use them for:

• Database connection strings

• API keys (encrypted with KMS)

• Feature flags

• Stage identifiers (prod, staging)

import os 
DB_HOST = os.environ['DB_HOST'] 
API_KEY = os.environ['API_KEY']

Permissions (Execution Role)

Every Lambda function needs an IAM execution role. By default, Lambda creates one with CloudWatch Logs permissions. You add policies for whatever the function accesses — S3, DynamoDB, SQS, etc.

Scenario: Your Daily-tenant-report function needs to read from DynamoDB and send emails via SES → attach AmazonDynamoDBReadOnlyAccess and AmazonSESFullAccess policies to the execution role.

VPC Configuration

Connect your Lambda to a VPC to access private resources like RDS databases or ElastiCache. When enabled, Lambda creates ENIs in your specified subnets.

Trade-off: VPC-connected functions may have slightly longer cold starts, though AWS has significantly improved this.

Function URL

Assign an HTTPS endpoint directly to your Lambda — no API Gateway needed. Great for simple webhooks or internal tools.

Triggers

Lambda can be triggered by 200+ AWS services:

• API Gateway (HTTP requests)

• S3 (file events)

• DynamoDB Streams (data changes)

• SQS/SNS (messages)

• EventBridge (scheduled/event rules)

• CloudWatch (alarms)

Destinations

Configure where successful or failed async invocation results go — SQS, SNS, Lambda, or EventBridge.

Concurrency and Recursion Detection

Concurrency simply means:

👉 How many times your Lambda function can run at the same time

• Reserved concurrency: Guarantees a set number of concurrent executions

  • Think of this as:

    👉 “I want to reserve a fixed number of slots for my function”

    • Guarantees that your function always has capacity available

    • Prevents other functions from using all resources

    📌 Example:

    • You set reserved concurrency = 10

    • Your function can run up to 10 times simultaneously

    • Even if the system is busy, these 10 slots are reserved for you

    ✔️ Useful for:

    • Critical applications

    • Preventing overload on downstream systems (like databases)

• Provisioned concurrency: Pre-initializes execution environments to eliminate cold starts

  • Normally, Lambda may take a little time to start (called cold start).

    Provisioned concurrency means:

    👉 “Keep some instances of my function already running”

    • Removes cold start delays

    • Improves response time

    📌 Example:

    • You configure 5 provisioned instances

    • These are always ready → faster execution

    ✔️ Useful for:

    • APIs

    • User-facing applications

    • Low-latency requirements

• Recursion detection: Prevents infinite loops where Lambda triggers itself

  • This is a safety feature.

    👉 Prevents your Lambda from calling itself again and again in a loop.

Code Signing

Ensures only trusted, signed code runs in your function. You create a Code Signing Configuration linking to an AWS Signer signing profile.​

Monitoring and Operations Tools

Lambda integrates with CloudWatch Logs, X-Ray (tracing), and CloudWatch Lambda Insights for performance monitoring.

Versions and Aliases

Versions

A version is an immutable snapshot of your function's code + configuration. When you publish a version, Lambda assigns it a number (1, 2, 3...). The $LATEST version is always mutable — it's your working copy.​

Scenario: You deploy v1 of Daily-tenant-report to production. You make changes and publish v2. If v2 has a bug, v1 still exists untouched.

Aliases

An alias is a named pointer (like prod, staging, dev) to a specific version.

aws lambda create-alias \
  --function-name Daily-tenant-report \
  --name prod \
  --function-version 5

Why aliases matter: Your API Gateway integration points to the alias ARN, not a version number. When you deploy v6, just update the alias — no need to change API Gateway.​

Additional Resources Explained

Layers

A layer is a .zip archive containing libraries, custom runtimes, or other dependencies. Instead of bundling everything in your deployment package, you attach shared layers.​

Scenario: Multiple Lambda functions use the pandas library. Create one layer with pandas, attach it to all functions. Update the layer once, and all functions get the update.

Each layer version is immutable and identified by a unique ARN.​

Event Source Mappings (ESMs)

An ESM is a Lambda resource that polls stream/queue-based services and invokes your function with batches of records.

Supported sources: SQS, Kinesis, DynamoDB Streams, MSK (Kafka), Amazon MQ, DocumentDB.

Scenario: An SQS queue receives order events. An ESM polls the queue and invokes your Lambda with batches of 10 messages. You configure batch size, batching window, retry policies, and parallelization.​

Capacity Providers (New — Lambda Managed Instances)

This is a new feature that lets Lambda functions run on EC2 instances managed by Lambda, combining serverless development experience with dedicated compute.

You create a capacity provider specifying VPC, subnets, security groups, IAM roles, and optionally instance types and scaling config. Functions using capacity providers get access to specialized EC2 instance types while Lambda still handles scaling and patching.​

Use case: Workloads needing GPU instances or specific hardware that standard Lambda doesn't offer.​

Code Signing Configurations

Ensures deployment integrity — only code signed by approved developers/CI pipelines can be deployed to your functions.

Replicas

Lambda@Edge replicas — when you associate a Lambda function with CloudFront distributions, AWS replicates your function to edge locations globally for low-latency execution.

Container Image Functions (Deep Dive)

Instead of uploading a .zip file, you can package your Lambda function as a Docker container image (up to 10 GB uncompressed) stored in Amazon ECR.

Three Ways to Build Container Images

1. AWS base image: Pre-loaded with runtime + runtime interface client. Easiest approach.​

2. AWS OS-only base image: Amazon Linux with just the OS. You add your runtime. Used for Go, Rust, or custom runtimes.​

3. Non-AWS base image: Alpine, Debian, or any custom image. You must include a runtime interface client.​

Example Dockerfile (Python)

FROM public.ecr.aws/lambda/python:3.12
COPY requirements.txt .
RUN pip install -r requirements.txt 
COPY app.py . 
CMD ["app.handler"]

Deploy a Container Image Function

# Build and push to ECR
docker build -t daily-report .
docker tag daily-report:latest 076829085184.dkr.ecr.us-east-1.amazonaws.com/daily-report:latest
docker push 076829085184.dkr.ecr.us-east-1.amazonaws.com/daily-report:latest

# Create Lambda function
aws lambda create-function \
  --function-name Daily-tenant-report \
  --package-type Image \
  --code ImageUri=076829085184.dkr.ecr.us-east-1.amazonaws.com/daily-report:latest \
  --role arn:aws:iam::076829085184:role/lambda-execution-role

When to Use Container Images vs .zip

• Container images: Complex dependencies, large packages (ML models), existing Docker workflows, need for custom OS packages

• .zip archives: Simple functions, quick iterations, smaller codebases

Important: You cannot change deployment type after creation — a container image function stays container, a .zip stays .zip.​

Function Lifecycle for Container Images

After uploading, Lambda optimizes the image (function is in Pending state). Once Active, it can receive invocations. If unused for weeks, it goes Inactive and requires re-optimization on next invocation.​

Advanced: SAM Template with API Gateway + Lambda

What is AWS SAM?

AWS SAM (Serverless Application Model) is a tool that helps you define and deploy serverless applications using simple configuration files.

Instead of manually creating:

• Lambda functions

• API Gateway

• IAM roles

• Event triggers

You can define everything in one file and deploy it together.

Think of SAM as:

“A simplified way to write CloudFormation specifically for serverless applications.”

Why use SAM?

Without SAM:

• You manually create resources from AWS Console

• Difficult to manage and replicate

With SAM:

• Everything is written as code

• Easy to version control

• Easy to reuse across environments (dev, prod)

Step 1: Install SAM CLI

SAM CLI is the tool used to build and deploy your application.

# macOS
brew install aws-sam-cli

# Linux
pip install aws-sam-cli

Step 2: Initialize a SAM Project

sam init --runtime python3.12 --name daily-report-api

This creates a project structure like:

• template.yaml → Main configuration file

• hello_world/ → Lambda code

• tests/ → Unit tests

• events/ → Sample test events

Step 3: Understanding template.yaml

This is the most important file.

It defines:

• Lambda functions

• API endpoints

• Database

• Permissions

Basic Structure

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

This tells AWS:

• This is a SAM template

• Use serverless transformation

Globals Section

Globals:
  Function:
    Timeout: 30
    Runtime: python3.12
    Architectures:
      - arm64

This applies default settings to all Lambda functions.

Meaning:

• Every function will use Python 3.12

• Timeout = 30 seconds

• Architecture = arm64

This avoids repeating configuration again and again.

Lambda Function Definition

DailyTenantReportFunction:
  Type: AWS::Serverless::Function

This creates a Lambda function.

Key Properties Explained

CodeUri: src/
Handler: app.lambda_handler

• CodeUri → Where your code is located

• Handler → Entry point of your function

MemorySize: 256

• Allocates memory to Lambda

• More memory = faster execution

Environment:
  Variables:
    DB_TABLE: TenantReports
    STAGE: production

• Environment variables for configuration

• Used inside your code

Permissions (Policies)

Policies:
  - DynamoDBReadPolicy:
      TableName: !Ref TenantReportsTable

This allows Lambda to:

• Read from DynamoDB table

- SESCrudPolicy:
    IdentityName: "reports@shipsy.io"

Allows Lambda to:

• Send emails using SES

Event Triggers (Very Important)

This is where SAM becomes powerful.

API Gateway Integration

Events:
  GetReport:
    Type: Api
    Properties:
      Path: /reports/{tenantId}
      Method: get

This means:

• Create API endpoint

• When someone calls:

  /reports/{tenantId}
  

• Lambda will run

Another API Endpoint

GenerateReport:
  Type: Api
  Properties:
    Path: /reports/generate
    Method: post

Now you have:

• GET API → fetch report

• POST API → generate report

Scheduled Trigger (Cron Job)

DailySchedule:
  Type: Schedule
  Properties:
    Schedule: cron(0 6 * * ? *)

This runs Lambda:

• Every day at 6 AM UTC

This is similar to your current EventBridge setup.

DynamoDB Table

TenantReportsTable:
  Type: AWS::DynamoDB::Table

This creates a database table.

KeySchema:
  - AttributeName: tenantId
    KeyType: HASH
  - AttributeName: reportDate
    KeyType: RANGE

• tenantId → Partition key

• reportDate → Sort key

BillingMode: PAY_PER_REQUEST

• No need to manage capacity

• Pay only when used

Outputs (Important)

Outputs:
  ApiEndpoint:
    Value: !Sub "https://\({ServerlessRestApi}.executeapi.\){AWS::Region}.amazonaws.com/Prod"

After deployment, this gives:

• Your API URL

Step 4: Build and Test Locally

sam build

• Prepares your application

sam local invoke DailyTenantReportFunction --event events/test.json

• Runs Lambda locally

sam local start-api

• Starts local API server

Test using:

curl http://localhost:3000/reports/tenant-123

Step 5: Deploy to AWS

sam deploy --guided

This will:

• Ask for configuration (region, stack name)

• Upload code to S3

• Create all resources

After Deployment

You will get an API like:

https://abc123.execute-api.us-east-1.amazonaws.com/Prod/reports/{tenantId}

Example API Calls

curl https://.../reports/tenant-456

Fetch report

curl -X POST https://.../reports/generate \
  -H "Content-Type: application/json" \
  -d '{"tenantId": "tenant-456"}'

Generate report

Final Architecture

Client → API Gateway → Lambda → DynamoDB ↓ Scheduled Event (cron)