This blog post is part of the Mixmax 2017 Advent Calendar. The previous post on December 4th was Terraforming All The Things.
Scaling recruiting
We want to fill our new office with stellar engineers. Let us know if you’re interested ;).
We've partaken in hackathons and careers fairs throughout the country where we've met hundreds of students who are interested in working with us. Simultaneously, we've been fortunate enough to receive a record number of applications from folks around the world.
We're thrilled that so many engineers are interested in working with us! Like any startup, though, time is our most valuable resource and we must spend it carefully.
Our recruiting process must be highly efficient so that we can find the best candidates, ensure our candidates have the best-possible experience when interviewing with us, and maintain high engineering velocity.
This year, as our candidate pool swelled, our process became excessively time-consuming and we decided that we needed to make a change.
Based on what we had heard from friends and engineers at other companies, we thought that using an online coding challenge would save us a lot of time. We started setting goals and got to work!
Designing the challenge
Beyond being an effective screen, we established that we wanted the challenge to:
Resemble real-world work
At Mixmax, we believe that it's important for our technical interviews to reflect the sort of work that engineering candidates would do with us day-to-day.
To ensure that the challenge would resemble the work we do, we decided to base the challenge on a problem we had actually solved in the product. In addition to making the challenge a more effective screen, we also hoped that this would make the challenge fun for candidates!
Support test cases
We wanted to let candidates test their code so that they could verify their work before submitting it.
Be easy to use by candidates
Some coding challenge platforms have hard-to-understand UIs and idiosyncratic testing mechanisms. We want candidates to feel as comfortable as possible when interviewing with us so that they can focus on the problems at hand; ease-of-use is an important part of achieving that comfort.
Not be excessively time-consuming
While we want to ensure that we're holding candidates to a high bar, we also want to be mindful of our candidates' time. We wanted the coding challenge to take candidates no more than 45 minutes.
Be available in a variety of programming languages
At Mixmax, we hire based on trajectory and demonstrated ability to learn rather than experience with the components of our stack. We believe that anyone we hire should be able to pick up new languages, frameworks, and technologies relatively quickly (and we'll need them to as our stack expands!). With that in mind, we wanted to make sure that we wouldn't filter anyone out simply because their language of choice was not available.
Picking Qualified
With those requirements in mind, we set out to pick a platform for our challenge!
We considered several options before settling on Qualified. We picked Qualified for a number of reasons:
Intuitive UI
We felt that Qualified's UI was intuitive and similar to a number of online editors with which candidates might already be familiar, like JSFiddle.
Test cases defined using real frameworks
Some of the platforms we considered required that test cases be defined with a single string input and a single string output. This was not compatible with the challenge we wanted to use and we were not interested in writing a string parser.
Other platforms were more flexible, but still used custom testing frameworks.
Qualified, on the other hand, uses popular testing frameworks for most popular languages, which made writing tests easy for us and resulted in the most real-world experience for candidates.
Awesome team
As we trialed Qualified, we found their team to be delightfully responsive to our feedback. Working with them has been a pleasure. Thanks, Qualified! :)
In practice
After selecting Qualified, we built the challenge.
We framed its description with a story of how the candidate arrived at this problem in their Mixmax work to share more about what it's like to be an engineer here.
We decided to let candidates take responsibility for testing their own code. We share a basic test case to help them get started, but leave them to test the full set of requirements.
We ask every candidate who takes the challenge to share any feedback they have. Thanks, candidates who have shared feedback! You've helped us improve our recruiting process :).
Results
Hundreds of candidates have taken our online coding challenge. It's saved us a ton of time and, according the feedback we've heard, many candidates have enjoyed it!
I had a lot of fun on the Mixmax coding challenge :D. It was so refreshing! I broke out my Jupyter notebook and just enjoyed myself for 45 mins :D.
Great knowing this is a reflection of daily work at Mixmax, it was just the type of work I was expecting and I loved it!
I have taken tons of coding challenges in the past few weeks, and this one really stuck out to me because it was a really interesting question to think about and one that actually seems relevant to the role I’m applying for.
Let us know if you'd like to give the challenge a shot! :)
