Hiring is Hard

The tech sector's hiring process has become something of a hot topic in the last few years. For the most part, everyone seems to agree that the current methods of technical hiring are ineffective and generally bad for both the prospective employee and employer. A lot of established companies are experimenting with ways to improve the "standard process", and a number of start-ups have appeared that are specifically trying to solve the problem of hiring.

One thing that everyone seems to agree on is that resumes are broadly terrible. Most people struggle to write good ones, and they rarely provide useful information to the employer - this is what spurred me to write an article on how to write a great resume. The fact of the matter, though, is that resumes can be a hard way to get your foot in the door. Aline Lerner wrote an interesting post on a small experiment she ran regarding resumes, which showed that they commonly failed to predict a candidate's success for her group. It was a very localized experiment, and the sample size was small (51 resumes used in total, 6 resumes used per participant), so it doesn't support a larger extrapolated conclusion, but it's a good read. Interestingly, in a later study Aline did, the results showed that having little to no grammatical errors in a resume is a strong predictor of candidate success (which I also stress in my blog post).

The goal of a resume is to get you to the next step, which is a technical interview of some sort. One of the start-ups working in this space, 'Coding Hire', has a pretty good blog post on the various forms of technical interviews. Here's the taxonomy of technical interview types I'll use going forward:

  • Lunch Meeting: Pretty straight-forward. Meet up with some people over coffee or lunch and have a 1-2 hour technical interview in the process.
  • 1099 Method: This refers to the practice of paying someone as a contractor to evaluate their performance before bringing them on full-time. The name comes from the 1099-MISC form that you receive as an independent contractor for tax purposes.
  • Code Review: A candidate is evaluated based on the open-source code they have contributed to projects or pushed to repositories on the web (e.g., Github).
  • Homework: The employer provides a take-home project that the candidate completes and then submits to the employer for assessment.
  • Whiteboard Interview: Spend a day solving technical questions on a whiteboard with a rotating group of interviewers. This is the current "Standard Process".
  • Phone Interview: Also pretty straight-forward: someone from the potential employer calls the candidate and asks technical questions. I have only seen this used as a technical screen rather than a full-blown interview.
  • Video Conferencing: A format that's becoming more common - the employer and candidate link up over a video call with screen sharing, and the candidate codes for the interviewers.

Whiteboard interviewing is the most widely used method, and is the "standard process" that has drawn so much ire recently. A candidate's ability to solve a complex technical design question, on the spot, without additional resources, in an environment that is very stressful for almost everyone, is understandably compromised. Consequently, it is difficult for candidates to demonstrate their true ability, and hard for the interviewer to judge a candidate's potential at the company.

Joel Spolsky, co-founder of StackOverflow, advocates for the whiteboard method in this post from 2006, noting that it's better to reject a good candidate than hire a bad one. The obvious problem with this is that you miss out on great candidates (called a "false-negative"). Many companies realize this, though, and their solution is to rely on you interviewing multiple times. As explained by Steve Yegge, this how Google and Amazon approach hiring. This approach commonly backfires, for obvious reasons, as demonstrated when the creator of the popular homebrew OSX program interviewed at Google:

The late Aaron Swartz was a fan of the 1099 method. Thomas & Erin Ptacek, on the other hand, note that the 1099 Method is a great way to send strong candidates to other employers - instead advocating for the Homework Method.

A company that specializes is hiring for YC-backed startups, Triplebyte, is experimenting with different methods. First, they tried screening with quizzes, followed by video conferencing, and now they are giving the Homework Method a shot. Interestingly, as a result of their first experiment, they decided that Fizz Buzz quizzes in screens and asking candidates about previous projects are both mostly a waste of time.

A number of the start-ups entering this area are tackling the problem with competitive online coding challenges, which is effectively the 'Homework Method', except it's supposed to be broadly applicable for many employers. HackerRank, TopCoder, and most recently Starfighter are great examples of this. Other startups are providing platforms to enable the Video Conference method - e.g., Coding Hire and Interviewing.io.

Unfortunately, many of these methods only work for specific areas of engineering. Writing code via video conference, for example, only really works for languages that are interpreted or relatively quick to compile. If you are an FPGA engineer writing RTL, coding in a video conference would be extremely difficult. It can take hours to compile an FPGA image, depending on the target, and you would need hardware on which to deploy the image to verify it. Simulation is an option, but there are complexities there that don't exist for "true software". And debugging would be nightmarish, either way. Many of these challenges would apply to the Homework Method for FPGA engineers, as well.

And what about hardware engineers? They generally go through whiteboard interviews, as well, but hardware design is a completely different beast from writing code. I can't imagine doing this remotely.

At Ettus Research, we have used the Code Review Method very heavily. We have hired 50% of our software & FPGA engineers based on their published open-source code and contributions to FOSS projects that we use. This method works great for engineers who have published code, as you already have a pretty good idea of the engineer's professional output. But it doesn't solve how you evaluate potentially awesome engineers without published code, or engineers who aren't in software.

I don't think we will find a grand new process that cleanly fixes technical hiring in the immediate future. That said, I think the industry is converging on some common themes when it comes to hiring methodologies. Here's a list that summarizes what I think are the common threads in all of the recent articles / blog posts / tech talks on the topic:

  1. Resumes, on their own, are a poor method of sourcing candidates.
  2. The best indicator of skill & talent is the candidate doing what they would do on-the-job.
  3. Whiteboard interviews likely cause you to miss out on great hires.
  4. Pedigree (e.g., school, previous employer) is not an indicator of talent.
  5. Cultural fit and technical communication skills are at least as important as technical talent.

There is obviously a lot more detail to the hiring process than I touched on in this post (e.g., interviewing techniques like behavioral interviewing, what to look for in candidates, what sorts of technical questions are worth asking, etc.,), but my primary assertion, here, is at a higher level than that. In short:

tl;dr: Hiring is hard.

Head here to see Part II of this post:
Update on Hiring: It's Still Hard

Ben Hilburn

Ben Hilburn

bits, nibbles, bytes, and words
D.C. Metro Area