Why LeetCode Actually Works (And Your "Realistic" Process Doesn't)

Your realistic interview process is unrealistically bad

Everyone hates LeetCode. I hate it; you hate it; we all hate it. Except, apparently, for Big Tech.

Fifteen years ago, I changed how we did interviews. I looked at what I and my peers actually did all day, and then I looked at what we forced candidates to go through. It just didn’t really connect. My hypothesis was that the closer the interview was to the real job, the more predictive it would be. If we improve our hiring process, we’ll build a stronger team.

And so, my crusade began. I badgered our Head of Engineering. Stalked our HR head in the hallways and annoyed our head of IT. Eventually, I got my way. A new proof-of-concept step in our interview process. Candidates would come into the office and get a laptop with a full IDE and a connection to the internet. They’ll work on a real task with the resources that they’d have if they joined.

I learned something that changed how I think about interviews.

If you’re designing your interview process right now, maybe you’re thinking of letting candidates use AI, pair program with the interviewer, or anything else to make it more like the real job. Well, you’re going to learn this too.

The impossible math of interviewing

Imagine you have access to a multiverse machine. And a time machine. Maybe a 2-for-1 combo machine you found on Amazon for Black Friday. The Universe Splitter 3000.

You can use this machine to actually solve hiring once and for all. Any candidate who applies for the job? Hire them! Send them off to some other universe to do the job and see how well they perform. Now take this data, go back across space-time to our universe, and hire the top performer.

Hiring. Solved.

Before we come crashing back down to reality, one question first. How long would you need to observe each candidate performing on the job? You’ll certainly want to see how they onboard first. You’re hiring for the long term, and not everyone ramps up as quickly. Maybe your best performer takes a while longer to ramp-up?

It’s common to use 30-60-90 ramp-up plans (please don’t!). So, let’s say 100 days is enough to make an informed decision. Let’s round it to 1,000 hours of real work. We’ll observe each candidate spending 1,000 hours working on our team and then choose the best one.

Alright, back here in reality.

You don’t have 1,000 hours to evaluate each candidate. You have 5 hours.

I assumed that if I simulated work, I could compress 1,000 hours of data in 5 hours, with very little information loss. If you stop and think about it, there really is no reason to assume this to be true. There’s no reason to assume that the more similar an interview is to real work, the more predictive it will be of success in the work. The order of magnitude just makes little sense.

You can’t win this game.

But you can change the game entirely.

What you can actually do in 5 hours

Okay, here’s what we know. We have five hours to make a judgment call. Here are three proven techniques that will help you build a great team:

1. Look at the inverse problem.

Don’t predict who will succeed on the job. Try to find who will be bad at the job.

Here’s an example. When I designed that interview process, one books that was fresh on my mind was Jon Bentley‘s Programming Pearls. In the book, he describes asking computer science Ph.D. students at Carnegie Mellon to write a binary search function. Not on the spot, but at home with no time constraints. Turns out that most students had a bug. Most? 90%. And it’s not just in academia; turns out it was true in the industry as well.

Spoiler. This statistic barely changes when people know about this grim reality before trying to do it. If you haven’t tried this yourself, I recommend trying to solve it on your own. These days, it means writing code somewhere with no LLM-powered autocomplete.

Now, there’s nothing inherent to imply that being able to write a binary search without bugs implies being good as a software engineer. The inverse is clearer, though. If you write buggy code in a well-defined problem, after being told most people write buggy code, then you will do so for the ambiguous problems at work.

Through this lens, the interview process is not about vetting. It’s about filtering.

2. Gather 1,000 hours of work in 5 hours.

You don’t have the Universe Splitter 3000. No multiverse for you. What you have is a time-machine. I mean, I wrote these words long before whatever the present is for you right now. You’re looking backward in time right now.

You can’t look forward in time, but you can look backwards.

This is the key insight behind Behavioral style interviews. You can do a 1,000-hour interview asking the candidate about the thousands of hours of work that they already did. Instead of inventing artificial and simulated scenarios, talk about real challenges that they faced. Ask them how they solved hard problems, navigated ambiguity, or considered trade-offs.

The real insight won’t come from the initial questions that you ask. It will come through the follow-up questions that dive deep and cut through the noise. This gets into the biggest challenge in integrating these interviews into your process. Running a good behavioral interview is hard, and you’ll need to invest in training your interviewers.

Once you have a good set of interviewers who have mastered asking piercing follow-up questions? It’s time to pick up the most under-used of all interviewing hacks: the reference calls.

Start calling references and interviewing them. This gives you the most confidence and data on those thousands of hours. It gives you another perspective on the candidate. Another not-very-objective perspective, which is why you need trained interviewers.

3. Run situational interviews

The research is pretty clear^[1]. Behavioral interviews outperform situational (simulation-based) interviews for knowledge workers. It’s also clear that combining both performs even better.

So how do you run a situational interview well? Well, first, don’t aim to simulate the work environment itself. Instead, work backwards from what success and failure look like, based on the historical data you have in your company. Then, design interviews that test these traits. By going upstream, you can get relevant data in a much shorter time.

Here’s an example. Let’s say you noticed that at work, the requirements sometimes change. Maybe the product manager changed their mind, Sales promised a non-existent feature that needs to be ready yesterday, or the business landscape changed. I’m sure nothing like this happens at your company, but I’ve heard it happens in other places. This means that any engineer who can’t handle these changes will fail. Now you have a specific trait to evaluate: adaptability.

You can evaluate adaptability in a situational interview. Give the candidate a programming task. Then, midway through, change the requirements. Do so in a way that breaks some implicit assumption they depended on until then.

Here’s a fun one that one of the Big Tech companies used to ask. Given two sorted arrays, find all the elements that are in both arrays. Seems like a super simple question. You can then throw in additional information, such as one array is too big to fit in memory.

This third example is a great segue to talking about LeetCode.

Why LeetCode works (when you use it right)

LeetCode questions really are terrible interview questions. Candidates who blindly grind through LeetCode can ace your interview questions and completely crash on the job.

LeetCode questions done well are great interview questions. Here’s what this means in practice:

• It’s not a binary test. Solving the question shouldn’t mean passing the interview. Failing the question shouldn’t mean failing the interview. Whether or not they solve it, it should be one data point.

• Work backwards. Coding speed is a perfectly reasonable evaluation criteria, for some companies. For others, it will have nothing to do with the traits that are upstream of success (or failure) on the job. Start with the traits and work backwards.

• Focus on signal collection, not decision making. This focuses the interviewer on the evaluation criteria. Their biased minds will jump to hire / no-hire decisions quickly. Instead of fighting this human shortcoming, accept it and make it as irrelevant as possible.

Most interviewers do the opposite.

They pick a problem they think is clever. Wait for the candidate to solve it. Then grade on completion.

That’s not interviewing. That’s testing trivia with extra steps.

The tool trap: from IDEs to AI

What happened 15 years ago when I let candidates use IDEs? The ones who used the tools did worse. They wasted time searching online instead of thinking. They threw minutes away fighting IntelliJ when they were used to Eclipse. Lots of time, very little data for us. Poor experience (and outcomes) for them.

Whiteboard questions are simple. No tech setup. No distractions.

Here’s an interesting data point. Meta uses Coderpad for remote coding interviews. They just turn off the ability to compile the code that you write. This surprises a lot of candidates. On the job, everyone can use a compiler. We already established that this isn’t really a useful consideration. Instead, the compiler is a distraction that provides little value.

Now we have AI. Should you allow AI in coding interviews?

My take is yes and no. You should have one coding round with LLMs allowed and one without. The evaluation criteria for the LLM-enabled interview should be different. The question design should be different. You just have to work backwards for the specific trait that you’re evaluating.

And you have to account for the churn and distraction. The coding environment will be more jarring for some candidates than others. This will mostly be based on their experience, with very little predictive power for the future. Right now, in late 2025, for most companies, this is still a worthwhile effort.

In 5-10 years? I think LLM-enabled interviews will be a waste of time.

Realistic feels better, artificial works better

I’ve written some controversial posts in the past. My guess is that this one will be my most controversial take. That’s fine. If it got you thinking more seriously about structuring interview loops, I’ll be happy.

And if you want to tell me just how wrong I am, please do.

P.S. One more thing. If you don’t work in Big Tech or one of the hottest startups, there’s a far more important lever to pull before you optimize your interview process. Get more qualified people to the top of your funnel. 9 times out of 10, you’re better off investing your efforts there.

---

Further reading:

Most teams engineer products better than they engineer teams

New Team, New Company, New Role. Here's Your Survival Guide