Defining the Role: Scorecards

Scenario

Example organisation: the platform team needs a "senior backend engineer". Each interviewer has a different picture: one wants Kubernetes expertise, another wants someone who can lead projects, a third wants strong algorithms. Debriefs turn into debates about what the role is; good candidates are rejected for different reasons by different people.

What you see on the team

• Job descriptions copied from the last posting.
• Interviewers who disagree about what "good" looks like.
• Hiring decisions driven by the strongest opinion in the debrief.
• New hires who don't match what the team actually needed.

The common mistake

Starting with a job description and a list of technologies instead of a scorecard describing the outcomes the role must achieve and the competencies that predict them.

Why it happens

Job descriptions are needed for posting, so they come first. But they're marketing documents; they don't define how to evaluate candidates. Without a shared definition, every interviewer evaluates their own idea of the role. Google's re:Work guide on structured interviewing is built on the same idea: decide the qualifications first, then evaluate every candidate against them consistently.

The playbook

Write a one-page scorecard before the job post:

Senior Backend Engineer - Platform team

Mission: make it easy for 12 product teams to ship safely.

Outcomes (first 12 months):
 1. Lead the deployment-pipeline migration for 48 services, with product teams migrating themselves.
 2. Reduce support requests by 30% through self-service tooling.
 3. Mentor two mid-level engineers to own platform components.

Competencies (and how we'll assess them):
 - System design and operability           -> design interview
 - Leading projects across teams           -> behavioral interview
 - Writing production code (Go or Python)  -> practical coding exercise
 - Communication with internal customers   -> behavioral + hiring manager
 - Mentoring                               -> behavioral

Not required: Kubernetes expertise (we'll teach it), specific cloud provider.

Agree it with the interview panel before the first interview.

Derive the job description from the scorecard: outcomes and competencies, not a laundry list of technologies.

Map each competency to one interview so nothing is assessed twice and nothing is missed.

What to say

Kickoff with the interview panel:

"Before we see candidates, let's agree what this person needs to achieve in their first year and the five competencies we'll assess. Each of you owns one or two — and anything outside the scorecard shouldn't drive the decision."

Trade-offs

• Writing a scorecard takes an hour or two; it saves far more in debriefs and mis-hires.
• Narrow scorecards can exclude good non-traditional candidates; list what's not required as well.
• Roles evolve; revisit the scorecard if the team's needs change mid-search.

Signals it's working

• Interviewers use the same language about candidates.
• Debriefs discuss evidence against competencies, not preferences.
• New hires' first-year outcomes match the scorecard.
• Job descriptions get better-matched applicants.

Key points

• Write a scorecard first: mission, first-year outcomes, competencies, what's not required.
• Agree it with the panel before interviews start.
• Map each competency to one interview.
• Derive the job description from the scorecard.