Hiring engineers has always been tricky, but AI has made it feel like a high-stakes cat-and-mouse game, with ChatGPT hiding behind the mouse. This post unpacks how AI is reshaping skills, enabling new “gaming” tactics, and why we need stronger, real-world hiring assessments.
AI Changed the Developer Skillset: Experience > Knowledge
AI coding tools now enable juniors to crank out boilerplate, and even full functions, boosting output by 27–39% for junior devs, versus 8–13% for seniors. But code quality still needs a human’s eye: only 42% of developers trust AI output, meaning most know that edge cases, security flaws, and architectural issues are best caught by experience.
As GitHub’s CEO said: “Startups can launch with AI code, but can’t scale without experienced devs”. Experience isn’t just nice-to-have; it’s the multiplier AI relies on.
AI Makes Cheating in Interviews (and Even Jobs) Easier
As tools like Cluely, backed by a $15 M Series A, openly market “cheat-on-everything” platforms, AI-powered interview fraud is becoming normalized. Cluely’s founder, who was famously kicked out of Columbia University for building an interview cheat tool, boldly predicted that using AI to cheat will soon be totally normal. He argues that “when every single person is using AI to cheat…, it’s just how humans will operate”.
You might have heard about the recent Soham Parekh saga, which reads like a Silicon Valley heist movie. The engineer allegedly held 3–6 full-time jobs at once, delivering top-notch interviews but little work, and sparked a storm in the startup world. Basically, he was overemployed, collecting multiple paychecks while doing minimal work on each job. Online communities celebrating “over‑employment” now boast 500K+ members, with one member boasting $800K/year in multiple roles.
This isn’t sci-fi; it’s real life. Slick resumes, AI-generated code, even real‑time AI help make it all too easy to fake credentials or juggle jobs undetected.
Testing Real Skills in Real(istic) Environments
The antidote? Real-world tests in realistic environments. Whiteboards and puzzles won’t expose AI-enabled bluffing. Instead, have candidates clone a repo, fix bugs, add a feature, using typical tools, and yes, allow AI.
70% of developers already use or plan to use AI in their workflow, and banning it creates a false filter. If you want to take it to the next level, have them communicate with (automated) colleagues, team leads, and support people.
This format lets you observe critical behavior: how they debug, AI‑prompt, validate results, and explain decisions. It separates true craftsmanship from autopilot copying.
The Take-Home Assignment Trade-Off
A full take-home project early on might save interviewers’ time, but it can also scare off top talent who aren’t yet invested - a six-hour project as a first step will push good candidates away. And if Company A asks for a weekend build vs. Company B’s two Zooms and offer, guess who wins?
A couple of ways to mitigate this:
- Transparency helps - explain the process early on and keep things predictable.
A shift toward short early filters + deeper later assignments improves engagement and interview fairness. But too late means the process isn’t scalable.
Make Hiring Challenges Great Again
A significant way to reduce candidate drop-off is to craft assignments that are engaging, relevant, even fun. Not only will you get better signal from motivated candidates, but you might actually enhance your employer brand in the process. It’s the difference between saying “here’s a test, don’t cheat, get it done by Sunday” and “welcome to our code challenge, have a blast with it and show us what you’ve got!”
You can structure the task like a game or puzzle where the candidate can immerse themselves and maybe even learn something new. When candidates are engaged, they’re far less likely to abandon the process. This can lead to a completion rate much higher than typical take-home tests. Why? Because it respects the candidate’s time and energy. It’s a win-win: you get to see their real skills, and they might actually enjoy the opportunity to show off those skills, and maybe even learn something new along the way.
Even if you don’t have fancy game platforms, you can make assignments more engaging by tapping into developers’ intrinsic motivation. Frame the task in a creative way: perhaps the candidate has to debug a “mystery bug” in a codebase (who doesn’t love a good bug hunt?), or build a mini-app that does something quirky or interesting rather than a sterile CRUD app. Some companies have had success with hackathon-style interviews or letting candidates choose one of a few challenges that pique their interest. The key is to show that you value the candidate’s experience, not just the output.
Also, consider giving useful feedback or reward for the effort. If someone spends time on an assignment, make it worth their while even if they don’t get the job. Maybe provide a cool insight, a discussion of their solution, or at least a thank-you email that isn’t just boilerplate.
Developers talk to each other, and if your hiring process is uniquely enjoyable or respectful, that word spreads. In a market where the best talent often chooses between multiple offers, a delightful candidate experience can tip the scales.
Conclusion: Hire for the Human, Not Just the Hacker
AI automates theory, but humans bring judgment, creativity, and accountability.
- Prioritize experienced intuition over textbook answers.
- Use real-world, AI-inclusive assessments that expose thinking, not copying.
- Design assignments to be realistic, fair, and yes, even fun.
It may feel unfair to lean on experience now more than ever, but the data’s clear. Experience is the difference between scaling and floundering in an AI-augmented world.
For junior engineers, the good news is that experience isn’t limited to time spent on the job. Today, hands-on learning environments that simulate real-world development can help build practical judgment, sharpen debugging instincts, and accelerate the growth of that elusive “senior mindset” — even before landing that first big role.
Here’s to hiring smart, creative engineers, and leaving the cheat codes behind.