Please dogfood your software interview homework

No one does the software homework they assign.

I’ve interviewed and held 5 different jobs over the past 8 years and at every single one I’ve had a similar experience. Most of my co-workers have barely read the homework, yet they are required to give feedback on candidates from it.

I understand everyone is busy, but the least you can do is eat your own dog food.

Experience 1 - Bad completion time estimates

When they issued me the homework they said it'd take 3 to 5 hours to solve and to not spend more than 10.

The homework was really difficult. It had me build an entire app, make a request to a weather api, and implement various UI components that were fairly new to android at the time. As 3 hours passed, I wasn’t even half way. I was getting anxious. Soon 5 hours passed and I felt like I had failed, but I pressed on.

Eventually I finished at 7 hours and submitted it.

After I was hired and had been working for a month or so, one of my co-workers had a question about something I had solved in the homework. As we hammered out a solution, I couldn't help but wonder, had they even attempted the homework?

I was frustrated because that specific part of the homework took me an extra 3 hours to solve.

I learned afterwards that no one had actually done the homework, not even my manager. They thought up a problem and guessed the time to complete it. Could they even solve it in 3 to 5 hours like they told me was possible?

Experience 2 - Too complicated of homework

At my next job, they had a infamous homework problem that almost all my co-workers said was hard to read.

Most of us didn’t have to do the homework because we were hired from a coding competition. We were often prompted for feedback on candidates' performance on it. I was never comfortable leaving feedback for candidates because I’d never done it myself.

So why did I never do the homework? Mainly because the problem was worded so weird, I never got started. On top of that, I didn’t want to get behind on my work tasks and my manager didn’t encourage it.

Despite knowing our homework problem was difficult to read, we kept using it for years. Some candidates would get partially done, others couldn’t even get started. Most of the time we passed on them.

I couldn’t help but think that had we required ourselves to do the homework how much better our hiring process could have been.

Experience 3 - The best candidates won’t do it

Just like driving, probably most people consider themselves to be a bit better than average. If you’re not willing to do your own homework yourself, how do you expect someone like you to do it?

The candidates you want are likely already happy in a job. They aren't going to spend 5 hours. They aren’t even going to spend 2. You’ll be lucky if they even respond to a recruiter.

I’m sure we’ll always need filters like FizzBuzz for the imposters. It’s hard to determine a great candidate because most of the ones we get can’t even code. Dealing with hundreds of resumes and candidates quickly gets unpleasant.

As an engineer, my first reflex is to automate things I don’t like to do.

The problem when automating homework, is there isn’t a good way to do it. What actually happens is you get something that is time consuming rather than a good filter. Remember good candidates are already happy where they’re at, they don’t have a lot of time.

How can we do better?

From watching how recruiters have been more open about job salaries, it seems companies are getting desperate.

In the sea of desperation, homework might be a thing of the past, as companies try to move faster.

But do we even need homework? I wanted to mention my favorite way I’ve interviewed over the past decade.

Do a ticket in the backlog

Pairing with my interviewer while I work on their codebase is by far my favorite way to interview. I’ve done it twice for 2 separate jobs. It easily simulates what it will be like to work at the company.

The pressure is still as intense as other formats like doing whiteboard problems. I think it’s way more comfortable though because I’m on an actual computer.

Both times I interviewed this way, I had zero experience in the language they used. I was upfront with them and received encouragement. I was able to solve the problem, while they occasionally helped me with syntax. I could explain why I was thinking of going a certain direction, mention trade-offs I was making, and google for things no one remembers how to do (like traversing a binary tree).

Most work is maintaining unfamiliar code

Making the interview about what I’m actually going to be doing (maintaining unfamiliar code) makes sense to me.

The first time I interviewed with a backlog ticket, I got the job.

The second time I didn’t. It was a startup you may have heard of called remix.run. Despite being rejected, I still feel like I was able to show way more of what I was capable of than any other format. They left the door open, so maybe I’ll work there someday.

Making better homework

If you don’t have the freedom to forgo the homework altogether, the next best thing you can do is serve it up as dog food for your current employees.

Who knows, they might actually learn something, or you can use it as an opportunity to teach.