Testing Python Functions with pytest and moto

Scenario

Example app: a Python function handles S3 uploads: it validates the file, writes metadata to DynamoDB and publishes an event. The team tests by uploading files to a shared dev bucket and reading logs.

What you see in production

Each change takes 20 minutes to test manually, so small fixes ship untested. A refactor that renamed a DynamoDB attribute reached production because nobody re-ran the manual test.

The naive code

import boto3
table = boto3.resource("dynamodb").Table("uploads")

def handler(event, context):
    record = event["Records"][0]["s3"]
    if not record["object"]["key"].endswith(".pdf"):
        return {"skipped": True}
    table.put_item(Item={"pk": record["object"]["key"], "size": record["object"]["size"]})
    return {"stored": record["object"]["key"]}

Reproduce it

Fast tests run the handler logic with sample events and an in-memory fake table:

class FakeTable:
    def __init__(self): self.items = []
    def put_item(self, Item): self.items.append(Item)

def make_handler(table):
    def handler(event, context=None):
        obj = event["Records"][0]["s3"]["object"]
        if not obj["key"].endswith(".pdf"):
            return {"skipped": True}
        table.put_item(Item={"pk": obj["key"], "size": obj["size"]})
        return {"stored": obj["key"]}
    return handler

def s3_event(key, size=10):
    return {"Records": [{"s3": {"bucket": {"name": "uploads"}, "object": {"key": key, "size": size}}}]}

table = FakeTable()
handler = make_handler(table)
tests = [("pdf stored", s3_event("a.pdf"), {"stored": "a.pdf"}),
         ("png skipped", s3_event("b.png"), {"skipped": True})]
for name, event, expected in tests:
    result = handler(event)
    print("PASS" if result == expected else "FAIL", name, result)
print("table:", table.items)

PASS pdf stored {'stored': 'a.pdf'}
PASS png skipped {'skipped': True}
table: [{'pk': 'a.pdf', 'size': 10}]

Root cause

When handlers create AWS resources at import and mix them with logic, the only way to run them is against real AWS. Manual, shared-environment testing is slow and flaky, so it happens rarely and late.

The fix

• Keep the handler thin; pass dependencies in (or create them through a small factory) so logic can be tested with fakes.
• Use pytest with sample events as fixtures (generate them with sam local generate-event s3 put).
• Use moto to emulate AWS services in-process for tests that exercise boto3 calls and data shapes:

import boto3, pytest
from moto import mock_aws

@pytest.fixture
def table():
    with mock_aws():
        ddb = boto3.resource("dynamodb", region_name="eu-west-1")
        yield ddb.create_table(TableName="uploads", KeySchema=[{"AttributeName": "pk", "KeyType": "HASH"}],
                               AttributeDefinitions=[{"AttributeName": "pk", "AttributeType": "S"}],
                               BillingMode="PAY_PER_REQUEST")

def test_pdf_is_stored(table):
    from app.handler import make_handler
    result = make_handler(table)(s3_event("report.pdf"))
    assert result == {"stored": "report.pdf"}
    assert table.get_item(Key={"pk": "report.pdf"})["Item"]["size"] == 10

• Add a small number of integration tests against a real ephemeral stack (per branch) for IAM, triggers and limits, and type-check with mypy or pyright.

Trade-offs

• moto is fast and convenient but not identical to AWS; keep real integration tests for permissions and service behaviour.
• Ephemeral stacks cost money and need automated cleanup.
• Over-mocking couples tests to implementation; prefer testing behaviour and data shapes.

Detect it in production

• Defect categories: logic bugs suggest missing unit tests; permission or wiring bugs suggest missing integration tests.
• CI duration and flakiness metrics.
• Event fixtures for every trigger a function handles.

Key points

• Thin handlers, injectable dependencies, sample-event fixtures.
• pytest + moto for fast tests of boto3 code.
• Real ephemeral stacks for IAM and wiring; type-check the code.

Further reading: moto documentation · Testing serverless functions