There’s often a gap between hiring best practices and hiring in the real world. In fact,in a post, the author has  shared some real-life stories of hiring gone wrong. In case you missed it, he tells a story from earlier in his career of when two senior engineers interviewed the same candidate and come away with radically opposed viewpoints.

As I began to dig into how this was possible, I came to a major realization: I had no idea what was actually happening in the interviews.

So I’d like to pose the question to you: Do you know what’s happening in interviews at your company? If your answer is yes, I’d follow that up with another question: How do you know that for sure? And if your answer is no, be sure to keep reading.

I’d like to share why it’s crucial for you to know what’s happening in interviews at your company and how you can get a handle on it if you don’t.

Crossed signals

In the case of the conflicting engineers I mentioned earlier, I learned that one person had his own pet brain-teaser he liked to use as a litmus test for all candidates. If he didn’t like the way they answered that question, he wouldn’t want to hire them. In effect, his method for evaluating candidates’ suitability was disconnected from the team’s goals and radically different from the criteria other interviewers were using (and in all honesty, probably not the best indicator of someone’s success on the job).

Learning this fact led to a significant realization: This was not a one-off problem. We’d always be talking past each other when we got to the round-up stage as long as we had an unstructured interview process.

A simple solution—and what it taught us

As soon as I came face-to-face with this lack of consistency in our interviews, I went and bought a bunch of digital recorders (this was in the pre-iPhone days) and asked our interviewers to record their interviews (with the candidate’s approval, of course).

And once we began to do this, the results were truly illuminating.

If you send your interviewers in without any sort of training or preparation, you’re bound to hear some surprising things—both good and bad. In our case, we were surprised by how creative our team could be. When candidates asked them about the company mission or values, some interviewers articulated this in ways that we had never anticipated.

On the other hand, we were shocked by how poorly some interviewers handled questions. When asked about the types of projects they were currently on, one interviewer said, “Eh, I’m not on a very interesting project right now.” They made no attempt to qualify this statement by saying that some past projects had been enjoyable or trying to learn more about which types of projects the candidate was hoping to get involved in. Hearing my own staff give these types of responses to candidates we desperately wanted to impress felt like a punch in the gut. But they helped us understand that we needed to provide much more in terms of training and coaching for interviewers.

A few final thoughts

Providing a framework for interviews and interviewers is a critical step in the Structured Hiring process and one of the cornerstones of the Greenhouse product. It’s also a work in progress for us. We’re always making adjustments to our interview planning, kits, and training. But none of this would have been possible without the insight I gained into the wild variance of the interview process. And I’d encourage you to do some research to see what’s happening in the interviews at your company—you’re bound to discover something interesting!