The Engineering Manager Handbook (Development Manager) · Hiring
Interview Loops and Calibration
Design an interview loop that covers every competency once, run debriefs on evidence, and keep the hiring bar consistent across interviewers and teams.
Scenario
Example organisation: candidates for senior roles go through seven interviews. Three of them ask about system design; nobody assesses mentoring. Debriefs take an hour and end with "let's do one more interview" for borderline candidates. Two teams hiring for the same level have very different bars.
What you see on the team
- Long loops with overlapping interviews and gaps.
- Candidates dropping out halfway; offers declined after slow processes.
- Debriefs dominated by one strong opinion.
- Inconsistent hiring bars between teams.
The common mistake
Designing loops by adding interviews until everyone feels comfortable, rather than by covering each competency once, and making decisions by debate rather than evidence against the scorecard.
Why it happens
Each additional interview reduces an individual's anxiety about a wrong hire. But more interviews make the process slower, harder on candidates and not more accurate when they overlap. Without calibration, every team drifts towards its own bar.
The playbook
Design the loop from the scorecard: one interview per competency group, typically 4–5 interviews in total for senior roles. Check coverage in a table:
| Interview | Competencies |
|---|---|
| Practical coding | Production code quality, testing |
| System design | Design, operability, trade-offs |
| Behavioral (cross-team) | Leading projects, communication |
| Behavioral (people) | Mentoring, feedback |
| Hiring manager | Motivation, role fit, the candidate's questions |
Set timelines: aim to complete the loop within one to two weeks; slow processes lose strong candidates.
Run debriefs on evidence:
- Everyone submits scores and notes before the meeting.
- Discuss competency by competency, quoting evidence.
- The hiring manager decides, using the scorecard; "one more interview" only if a specific competency lacks evidence.
Calibrate regularly: hiring managers and senior interviewers review a sample of decisions each quarter; shadow each other; align on what "good" means per level. A shared career ladder helps define levels consistently.
What to say
Opening a debrief:
"Scores are in. Let's go competency by competency — evidence first, then scores. If we're missing evidence on a competency, we'll say which one before suggesting another interview."
Trade-offs
- Shorter loops risk missing something; good coverage and rubrics matter more than length.
- Hiring manager decides keeps accountability clear but needs checks (calibration, a bar-raiser role) against individual bias.
- Fast timelines need interviewer availability planned in advance.
Signals it's working
- Loops have 4–5 interviews with no overlaps or gaps.
- Time from first interview to offer drops to one to two weeks.
- Debriefs take 30 minutes and cite evidence.
- Hiring outcomes are consistent across teams at the same level.
Key points
- Design the loop from the scorecard: each competency assessed once, 4–5 interviews.
- Move fast: complete loops within one to two weeks.
- Debrief on evidence, competency by competency; scores submitted beforehand.
- Calibrate across teams regularly, using shared level definitions.
Practice questions
A senior hiring loop has seven interviews, three of them on system design. What's wrong and how do you fix it?
Answer
Overlap and gaps: some competencies are tested repeatedly and others not at all, and the long process loses candidates. Rebuild the loop from the scorecard so each competency is assessed once in 4–5 interviews.
How should a hiring debrief run?
Answer
Scores and notes submitted before the meeting; discussion competency by competency with evidence; the hiring manager decides against the scorecard; another interview only to fill a specific evidence gap.
How do you keep hiring bars consistent across teams?
Answer
Shared level definitions (a career ladder), common rubrics, quarterly calibration reviews of decisions, and cross-team shadowing of interviews.
Open in the app to mark lessons complete, take notes and track your progress.