On Hiring: Beyond The Interview

There are no books you can read or blog posts you can scan that will guarantee that you make the right hire 100% of the time. From bad chemistry to misunderstandings about role expectations, even the strongest candidate may not work out. Also, despite best efforts, early stage companies or new teams inside scaling business often don’t know what they need until they have someone in a particular role. You might realize “oops, this person is great, but their skills are not what we need!”. It happens at every company. Hiring is HARD.

While you can’t prevent occasional mis-hires, you can try to minimize the possibility by including a project or testing phase in your hiring process. This is the stage beyond the standard interview questions and reference checks that gives you a sense of who this person really is, their skills and how they will approach their role. The goal of these tests is to allow the candidate to demonstrate what they are capable of and what it might be like to work with them once they are on board. These tests are critical and will either help you dodge a mis-hire or give you a higher degree of confidence that this is “the one”. I recommend that these tests are performed when you have 1–2 finalists and just before you are ready to do reference checks and make an offer. This can be an especially helpful step if you are down to two finalists you really like.

With this in mind, below are some tips on how to do these tests. For each of these tests, it’s about how the candidate approaches the test and the problem vs. getting correct answers. Build alignment with your team on what “good looks like” for each test and plan to debrief once the assignment is complete and/or presented. Examples of what good might look like are included below.

“The First 90 Days” Test

This is a good general test for any new hire, especially an executive, but also for a people manager or technical leader. Have the candidate explain what their first 90 days on the job will look like. Either leave it wide open or offer a few prompts like “Who will you spend time with?” or “How will you get to know the business?” or “What accomplishments do you hope to make by the end of the first 90-days?”. Avoid being overly prescriptive or leading questions like “Name all the team members you’ll want to get to know” or “Will you spend time with marketing and sales?”. An experienced candidate should have a good sense of how they would spend their first 90 days based on the research they’ve done on your company and insights they’ve gained from interviews with the team.

What good might look like:

• Thoughtful about talking with the right stakeholders and when — align with your team on who they’d expect to see on the candidate’s list and when they’d expect to meet with this new hire within the first 90 days
• Organized and realistic about what can be accomplished in 90 days — align with your team on what you’d expect
• Asks good questions to get a feel for the assignment — shows they are comfortable with getting clarity on situations (not arrogant)
• Articulates assumptions made (if any) — often a requirement of leadership roles and demonstrates strong communication skills
• Cites examples from conversations they’ve had with team members/research they’ve done on the company, market, etc. — demonstrates they listened, interested and have taken the time to understand the opportunity

Engineering and Design Tests

While there are some nifty tools out there that can test coding skills for engineers, I am a strong advocate for testing the human side of these skills. Those who design and/or build your product should be able to demonstrate their work beyond coding or portfolio samples. The best tests here are brief scenarios that demonstrate not just depth of syntax knowledge or design best practices, but also how they will work on a problem with your team. These types of tests can be done as “homework” although it’s nice if it can be done in-person as part of an onsite/video interview. Present a scenario and ask the candidate how they will approach it. You could give them some alone time to think it through and then ask them to talk you (or a domain expert) through it. Ask them to cite how they thought about it and explain the direction they took and why. Prepare to have another approach or idea for the scenario when they walk through their work. This can help gauge how the candidate handles feedback and if they are willing to collaborate on ideas.

Try not to give an assignment that takes more than 1–2 hours to do unless you pay them for the work. I know a company who always pays for the time taken to do the test and if the candidate declines payment, they make a donation to their charity of the candidate’s choice for their time. (So cool!)

What good might look like:

• Asks good questions to get a feel for the assignment
• Articulates assumptions made (if any)
• Able to explain their work and creative approach; approach aligns with how your team operates and/or offers new ideas that will expand the possibilities for your team/product
• Comfortable receiving feedback; bonus points if they’re willing to riff on the idea and take it to a better place.

Scenario Tests For Functional Teams (Marketing, Sales, Product, etc.)

Functional leaders are often asked to present a past project they did as a way to demonstrate their work. While this lends insight into the candidate’s past work, I prefer scenario tests. While the former does tell you an experience they had and what worked or not, it will not expose their work on something new. Further, a walkthrough of a former project may not give you insight into what they (vs. other team members) actually did to achieve success. In those cases, I listen for “we did this…” which begs the question “what part of that did YOU do?”. Here are some quick examples of scenario tests for a few functional areas:

Product: Our CTO just came back from a “listening tour” with some of our customers and wants to explore a new set of features to expand our product offerings. These offerings are not on the product roadmap. What steps would you take to understand these new features and how would you approach the prioritization process?

Marketing: We’re about to launch a new product for our customers. It is the first new product we’ve launched in a year. What steps would you take to plan for this product launch and how will you measure its success?

Sales Leader: We are building a product to attract new customers in a new vertical. What information do you need to prepare your team to sell this new product and how will you set sales goals for the team?

You could imagine similar scenarios for finance, customer support or other functional roles. Remember, they still don’t know how your business functions day to day and this isn’t whether they have a perfect plan, but more about how they approach the problem. For functional roles that will require strong communication and presentation skills, have them present their assignments as they would if they were doing it for your team, board or customers, depending on the scenario. For presentations, the ideal flow is interviews, assignment for finalists, and then a presentation to all those who interviewed the candidate. Other key stakeholders could sit in on the presentation, but to mitigate overwhelming the candidate, I suggest only those who interview them do Q&A after the presentation. Q&A should probe what’s being tested (what good looks like) and not to have candidates try to get correct answers.

What good might look like:

• Demonstrates research they’ve done to prepare the assignment
• Including people on your team they may ask to speak with to prepare their preparation
• Presentation skills — both oral and written. Focus more on content and less on pretty graphics on presentation decks unless that’s an important element of the role
• Articulates assumptions made (if any).
• Scenario solution is thoughtful, logical and realistic — align with your team in advance on what this would look like
• Bonus points if they add insights that the team can learn from (e.g., I once had a VP Marketing candidate tell us what was broken with our SEO and how to improve it as part of his presentation of a hypothetical scenario. It was brilliant!)

With all of the interviews and testing, you still may not get it right every time. Again, hiring is HARD. For some roles, a “try before you buy” is often the best way to go for both the candidate and the company. Not every candidate can opt out of benefits or other things they need from a full time job to do a trial, but if it’s possible or they can do it outside of work, go for it. Pick a project that’s reasonable to do in 30–45 days and agree on what good will look like before they get started. Pay them an hourly rate and set the candidate up for success so they can hit the ground running (e.g., security access to your code repo, slack, etc,) and continue to test the soft skills as they go. if applicable, tell the candidate in advance that if they are hired, equity vesting will start when they started their project vs their actual start date. It’s a nice incentive for them to take the project seriously and know you are invested in their success.

Finally, if you’re hiring a role for the first time and no one on your team has experience with that role — no one knows what good looks like — ask an advisor, investor or friend with experience to be part of the interview process. They should not only be able to interview the candidate, but also help you formulate the tests!

Do you have other tests or projects you like to use as part of the hiring process? Please share in the comments!