Sign in Get started

Coding problem · Easy · Language Idioms

Lazy chunked() Generator

Easy Language practicals Seen at Spotify, Dropbox, Instagram

Write a generator chunked(iterable, size) that yields lists of size items, with a shorter final list if needed.

  • It must be lazy: it should work on infinite iterators and on huge files without loading them into memory.
  • size < 1 raises ValueError (raising on first use is fine).

Typical use: batching database inserts or API calls.

Hints

Hint 1

itertools.islice(it, size) takes up to size items from an iterator.

Hint 2

Call iter() on the input once, then loop until a slice comes back empty.

Write and run your solution

Loading the editor… (JavaScript is required to run code in your browser.)

Solution and explanation

Show the solution

Time O(n) total · Space O(size)

iter() turns any iterable into a single iterator; islice(it, size) consumes up to size items from it each loop, and the walrus operator stops when a chunk comes back empty. Because the function contains yield, nothing runs until the caller iterates — which is why the ValueError surfaces on first use.

Python 3.12 added itertools.batched(iterable, n), which yields tuples — mention it, and that you'd still write this for older runtimes or when you need lists.

Reference solution

from itertools import islice
from typing import Iterable, Iterator, TypeVar

T = TypeVar("T")


def chunked(iterable: Iterable[T], size: int) -> Iterator[list[T]]:
    if size < 1:
        raise ValueError("size must be >= 1")
    it = iter(iterable)
    while chunk := list(islice(it, size)):
        yield chunk

Common mistakes

  • Building the whole list first (list(iterable)) — breaks on infinite or huge inputs.
  • Calling iter() inside the loop — restarts lists and repeats the first chunk forever.

Follow-up questions

  • Write a sliding-window generator.
  • Batch inserts to a database with chunked() and executemany().