In this episode, Jason Haddix (CEO of Arcanum Information Security and creator of the Bug Hunter’s Methodology) joins us to examine how AI is changing penetration testing and security research. He explains that while AI agents can automate reconnaissance, code analysis, and parts of vulnerability discovery, meaningful results still depend on human expertise, methodology, and context engineering.
The conversation explores how AI is shifting the entry path for new security practitioners, why deep research and critical thinking remain essential skills, and how experienced testers are embedding their knowledge into agent workflows using tools like Claude Code. Jason also discusses practical experimentation with AI assistants such as OpenClaw, including prompt-injection defenses, guardrails, and the operational risks of running autonomous systems.
The episode also addresses the growing debate around AI-generated code and AI-driven vulnerability discovery, highlighting the difference between marketing claims and real-world results. It closes with a discussion on why the industry needs better benchmarks and evaluation methods to measure whether AI security tools actually find meaningful vulnerabilities.
00:00–02:14 — Introduction to Jason Haddix and how his journey from bug hunter to Arcanum founder shapes his perspective on AI in security
02:14–08:00 — How AI agents are beginning to automate penetration testing workflows while still relying on expert methodology
08:00–10:45 — Why human expertise remains critical even as security automation improves
10:45–17:10 — How AI is changing the learning curve for the next generation of pentesters
17:10–25:27 — How agent frameworks and skills are transforming security tool building
25:27–35:41 — Security risks and defenses when running AI assistants like OpenClaw
35:41–40:32 — The rise of AI-powered personal assistants for research and security workflows
40:32–42:55 — Why the cybersecurity community is rapidly adopting AI tools
42:55–46:42 — How AI improves security coverage and turnaround time at scale
46:42–50:31 — Why newer models like Opus 4.5 unlocked practical AI security workflows
50:31–56:48 — The debate on whether AI should generate secure code or detect vulnerabilities
56:48–01:01:18 — Why AI security needs better evaluation benchmarks and real-world testbeds
Tune in for a deep dive!
Connect with Jason Haddix:
LinkedIn: https://www.linkedin.com/in/jhaddix/
Connect with Anshuman:
LinkedIn: anshumanbhartiya
X: https://x.com/anshuman_bh
Website: https://anshumanbhartiya.com/
Instagram: anshuman.bhartiya
Connect with Sandesh:
LinkedIn: anandsandesh
X: https://x.com/JubbaOnJeans
