Most job interviews for computer programmers are done poorly

(written by Lawrence Krubner, however indented passages are often quotes)

I think this is very true:

A candidate would come in, usually all dressed up in their best suit and tie, we’d sit down and have a talk. That talk was essentially like an oral exam in college. I would ask them to code algorithms for all the usual cute little CS problems and I’d get answers with wildly varying qualities. Some were shooting their pre-canned answers at me with unreasonable speed. They were prepared for exactly this kind of interview. Others would break under the “pressure”, barely able to continue the interview.

To be honest, when we first started doing this I had to look up these puzzles in advance, mainly to make sure I didn’t embarrass myself. This should have been the first warning sign that maybe we weren’t exactly testing for skills that were most relevant to our requirements. If these doubts occurred to me, I must have dismissed them very quickly. After all, it was the way everyone approached the interview process.

Of course, we ended up hiring the candidate with the smoothest answers. Inevitably, the next job openings came, we did it again and again in the same fashion, for the rest of the company’s lifetime. If this sounds familiar to you, you are clearly not alone.

Actual Job Performance
‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
But how did the candidates we selected measure up? The truth is, we got very mixed results. Many of them were average, very few were excellent, and some were absolutely awful fits for their positions. So at best, the interview had no actual effect on the quality of people we were selecting, and I’m afraid that at worst, we may have skewed the scale in favor of the bad ones.

What does bad and good even mean in this context? Let’s have a look some of the benchmarks that I consider important:

Company Culture. In hindsight, one of the most important features a new employee should have is compatibility with the spirit of the people who already work there. The Standard Dev Interview performed worst in this area, for obvious reasons. It’s difficult to judge people’s personalities in interviews because they are not exactly themselves. In fact, they’re incentivized not to be themselves.

Programming Competence. Somewhat counterintuitively, the code examples done during the interview were a bad indicator for actual competence on the job. Real world projects rarely consist of implementing binary searches without access to a parser or literature. It turned out that employees who had done very well in the code examples were not always able to transfer theoretical knowledge into practical solutions. Having candidates write sorting algos on the whiteboard is a method that rewards people with great and precise short-term memory who come prepared for exactly these kinds of questions. In our case, we needed resourceful coders who could write neat, stable, and elegant software – and the interview process wasn’t delivering them to us.

I’d add a suggestion, which is that if you are interviewing a computer programmer, you should take them to a meal — breakfast, lunch or dinner, it does not matter, but take them outside the office. That way you get to see how they interact with people in real life, and that gives you a wealth of information about how they will really behave with other members of the team. Just go outside the office. Going to a restaurant and ordering lunch is a great exercise in testing social skills.