An engineer hears critical feedback for the first time in her annual review. What went wrong?

The manager didn't give timely feedback during the year; significant feedback should be discussed in 1:1s when it happens so reviews contain no surprises.

What is calibration and why does it matter?

Managers reviewing proposed ratings together, comparing evidence, so that the same level of performance gets the same rating regardless of manager; it reduces individual bias and inconsistency.

How do you avoid recency bias in reviews?

Keep notes throughout the period — wins, feedback, projects, peer comments — tagged by ladder dimension, and write the review from that evidence rather than memory.

Performance Reviews and Calibration — The Engineering Manager Handbook (Development Manager)

Scenario

Example organisation: review season. Managers write reviews from memory in the last week. One manager rates everyone "exceeds"; another rarely gives above "meets". An engineer receives critical feedback in her review that she has never heard before and files a complaint.

What you see on the team

Reviews based on the last few weeks (recency bias).
Inconsistent ratings between managers for similar performance.
Surprises in reviews, leading to distrust.
Reviews that describe activity, not impact.

The common mistake

Writing reviews from memory, alone, at the last minute — instead of from evidence collected all year, checked against shared expectations, and calibrated with peers.

Why it happens

Review season is busy, and reviews feel like a once-a-year task. Without notes, memory favours recent and visible events. Without calibration, each manager's personal scale becomes the employee's rating.

The playbook

Collect evidence continuously — in your notes per report: wins, feedback given, projects, peer comments — tagged by ladder dimension.

No surprises: anything significant in the review should already have been discussed in 1:1s.

Write the review in a clear structure:

Summary: one paragraph on overall impact this period, at their level.
Impact with evidence: 3-5 examples, each with outcome and ladder dimension.
Growth: 1-2 areas to develop, with specific examples and a suggested plan.
Rating / level assessment: per company process, with the reasoning.

Gather peer input with specific questions ("what's one thing they should keep doing / do differently?").

Calibrate: managers review proposed ratings together, compare evidence for similar ratings, and adjust outliers. Watch for bias patterns (who gets "exceeds", who gets vague feedback).

Deliver in conversation, not just as a document: walk through it, listen, agree next steps.

What to say

Opening the review conversation:

"Nothing in here should be a surprise — we've talked about each of these. Let's go through the impact first, then the growth area, and then I'd like to hear your view and agree goals for the next half."

Trade-offs

Continuous notes take a few minutes a week; they make reviews faster and fairer.
Calibration takes time and can feel political; focus on evidence and consistent standards.
Ratings create anxiety; spend most of the conversation on impact and growth.

Signals it's working

No review contains feedback the person hears for the first time.
Ratings distributions are consistent across managers for similar evidence.
Reviews cite specific outcomes and ladder dimensions.
Engineers describe the process as fair.

Key points

Collect evidence all year, tagged by ladder dimension.
No surprises: discuss significant feedback in 1:1s first.
Structure reviews around impact with evidence and specific growth areas.
Calibrate with peer managers and check for bias patterns.