Tech Interviews in the AI Era
It's plainly obvious to all that the use of AI software development tools is fundamentally reshaping the role of the software engineer — with implications that traditional technical interview techniques must also be transformed.
"CTO Lunches" is an online forum for heads of engineering and technical leadership where folks share best practices and debate the nature of their practice. I've summarized a recent robust email thread under the subject "Tech interviews in the AI era." The conversation revealed that while most companies are still running some version of LeetCode-style interviews, a meaningful cohort of engineering leaders — predominantly at startups and growth-stage companies — have already overhauled their process. The rest are watching closely.
Auditioning for Software Development
One comment particularly stood out: a shift in their approach from "interviewing" to "auditioning" the candidate. It's a subtle terminology change, but it alludes to a broader shift that engineering teams should consider.
The shift from "interviewing" to "auditioning" moves the process from a theoretical assessment to a practical, high-fidelity performance simulation. Interviewing traditionally involves abstract problem-solving, structured Q&A, and potentially low-fidelity tests that assess what a candidate knows. Auditioning, as described in their circumstance, involves a high-fidelity simulation of real, everyday work. It focuses on how a candidate performs the actual job duties and integrates with a team, providing a more relevant and direct measure of on-the-job competency.
We're more familiar with the notion of auditions in acting for TV, film, or theater. Actors audition rather than interview because the ability to perform the role is the job requirement. An audition allows a director to see the person in the present tense — how they embody the character, their choices, and their range — which is difficult to capture through credentials or Q&A. In the same way, the technical "audition" assesses the engineer's ability to perform the job in a realistic simulation.
Auditions are also a chance to see the actor's skills and choices as a co-creator. The decisions an actor makes in presenting a monologue — what material to choose, the performance choices — reveal their ability to collaborate and bring something unique to the table. This is key for finding creative talent, and an even more important soft skill in what is emerging as Agentic Engineering: a model of software development where engineers increasingly orchestrate AI agents to plan, write, and debug code rather than doing so entirely by hand.
New Competencies for an Agentic World
As technical teams shift into new forms of Agentic Engineering, some are modifying their recruitment process to evaluate candidates on new, AI-driven competencies.
Assessing High-Level Skills
Addressing Candidate Deception
A significant number of interviewees are suspected of not being who they claim to be due to sophisticated AI deception tools like auto-generated resumes and fake video backgrounds. This raises its own uncomfortable tension: if a solution is to abandon open submissions in favor of targeted outbound outreach, return to in-person interviews, or use identity vetting services, does that inadvertently disadvantage distributed or remote-first talent pipelines? Several forum members flagged this directly, and no consensus was reached. It is a real tradeoff the industry will need to resolve.
The Broader Shift: What Andrej Karpathy Calls "Understanding"
Andrej Karpathy has offered advice that centers on shifting the human role away from writing code — which can increasingly be outsourced to agents — toward critical judgment and system ownership. His framework is worth quoting directly in spirit: you can outsource the thinking, but you cannot outsource your understanding.
The new skills, as he describes them, include owning the spec, design, and judgment calls; orchestrating agents on large, complex projects; and beginning every project with strong specification and architectural definition, since the directions given will determine future outcomes. The bottleneck, in other words, is shifting from coding to decision-making. And engineers remain fully accountable for their software — including developing taste for code quality, security, and architecture, and knowing how to vet outputs for security pitfalls.
What This Means for Candidates
The piece so far has been largely written from the hiring team's vantage point. But the implications for candidates preparing for the market are just as significant. Demonstrating fluency with AI tools is no longer a differentiator — it's becoming table stakes. What will differentiate candidates is the quality of their judgment when using those tools: their ability to write precise, well-scoped prompts; their capacity to critically evaluate AI-generated output rather than rubber-stamp it; and their willingness to own the final product rather than attribute its flaws to the agent.
Practically, this means candidates should practice directing AI agents on real, moderately complex projects — not toy examples. They should develop the habit of reviewing agent-produced code with genuine skepticism. And they should be prepared to walk an interviewer through not just what they built, but the decisions they made along the way: what they prompted, why, what the agent got wrong, and how they caught it.
The Way Forward
The practices emerging from forward-leaning engineering teams map closely onto Karpathy's framework. Reviewing a candidate's prompts and agent guidance directly tests orchestration — the ability to effectively lead and coordinate AI systems. Comprehension and troubleshooting tests mirror the requirement for understanding and accountability, forcing candidates to demonstrate the fundamental knowledge needed to vet outputs and fix the brittle results that statistical AI agents sometimes produce.
The shift toward design and architecture conversations aligns with the need for engineers to own specs, judgment calls, and high-level decisions. And the emphasis on clear communication throughout the hiring process reflects the new necessity for detailed, precise specifications — because in an agentic workflow, the quality of your language increasingly determines the quality of your software.
The audition metaphor, then, isn't just a rebranding of the interview. It reflects a genuine change in what engineering is becoming: less about producing code directly, and more about owning the vision, directing the systems, and being accountable for what comes out.
"}}}]}}},"pageContext":{"uid":"tech-interviews"}},"staticQueryHashes":["2046486224","3564657959","3649515864","63159454"]}