The Problem With How Most Interviews Work
Most interviews follow a loose format: the interviewer reviews the resume beforehand, asks questions that feel relevant in the moment, and forms an impression by the end. The research on this format is not favorable. Unstructured interviews have predictive validity scores in the range of 0.20 to 0.38 in most meta-analyses, meaning they are better than random chance but not by a large margin. Interviewers are susceptible to halo effects, affinity bias, and first-impression anchoring, all of which distort the signal they think they are getting.
Structured interviews, when implemented well, raise predictive validity to 0.50 or higher. That is a significant difference when multiplied across hundreds of hiring decisions.
What Makes an Interview Structured
A structured interview has three core components: standardized questions (every candidate for a given role is asked the same questions in the same order); defined scoring criteria (interviewers score responses against a rubric that specifies what a strong, adequate, or weak answer looks like); and independent scoring (interviewers record their scores before discussing with others on the panel, preventing groupthink and anchoring).
Building Questions That Actually Reveal Something
Structured interviews typically use behavioral questions, which ask candidates to describe past experiences, or situational questions, which present a hypothetical. Both formats outperform questions like "Where do you see yourself in five years?" which are so familiar that candidates have rehearsed non-answers for them.
Good interview questions require a specific example or specific reasoning, not a general statement. They relate directly to a competency required for the role, they cannot be answered well with a canned response, and they invite follow-up probing: "What was the outcome?" "What would you do differently?" "How did you make that decision?"
Designing a Scoring Rubric
A rubric anchors the evaluation to observable evidence rather than impression. For each question, you define what a strong response includes, what an adequate response includes, and what a weak or missing response looks like. Write these anchor descriptions before you start interviewing, not after. Once you have heard a compelling candidate, the temptation is to reverse-engineer the rubric to fit what they said.
The Calibration Problem
Even with standardized questions and rubrics, interviews can drift if the interview panel is not calibrated to the same standard. Calibration means running a practice session where the whole panel scores a sample response independently, then comparing scores and discussing discrepancies. This surfaces interpretive differences before they affect real candidates.
Independent Scoring and Panel Debrief
The debrief is where structured processes most often break down. Collect scores in writing before the debrief begins. Start the debrief by having each person share their overall score simultaneously. Only then open discussion, starting with areas of disagreement rather than areas of consensus. This process surfaces genuine disagreements and forces the panel to engage with evidence rather than deference to seniority.