Contents
Get a Personalized Demo
See how Torq harnesses AI in your SOC to investigate, prioritize, and respond to threats faster.
John White is the Field CISO for EMEA at Torq. A respected security executive with more than 20 years of leadership experience, John previously served as CISO at Virgin Atlantic, where he led a multi-year transformation deploying the Torq AI SOC Platform to modernize cyber operations. Prior to Virgin Atlantic, he built and transformed security functions for global organizations, including ASOS, Liberty Global, AEG Europe, and KPMG.
I’ve spent more than 20 years in security, and a good chunk of that time was spent in the buyer’s seat — sitting across from a vendor during a proof of concept, trying to work out whether the technology in front of me would actually solve the problem I had or turn into another integration project that ate more time than it saved. I’ve run those evaluations at Virgin Atlantic, ASOS, and Liberty Global. I know what a good one looks like, and I know how they go wrong.
So I’ll say this plainly: how to evaluate an AI SOC is not a question your old process can answer. The evaluation criteria we all relied on in the SOAR era, the SIEM era, the MDR era? They don’t fit anymore. And if you run this purchase the way you ran those, you’re going to end up with the wrong platform and a very expensive lesson.
The Category Outran the Frameworks
A year ago, a handful of vendors claimed the AI SOC category. Today, more than a hundred do, and almost every one has bolted “agentic” onto its positioning. Analyst firms are redrawing the category boundaries in real time, and the noise in the market is deafening.
Most of it doesn’t hold up. A lot of what gets sold as an “AI SOC” either stops at triage, wraps a chatbot around a legacy stack, or hides its reasoning in a black box. The plainest test I know still cuts through all of it: if it can’t take action across the threat lifecycle, it isn’t an AI SOC.
And the pressure to decide has only gone up. Attackers are moving at machine speed. The CISOs who were cautiously exploring AI-driven security operations last year are now being asked to commit, with budgets approved, timelines set, and boards expecting results. There’s no longer a long window to plan, adopt, and evaluate. Take the slow and considered path, and the technology will have moved on before you reach a decision.
Why the Old RFP Fails
The first sign that a CISO is still using the old rubric is the questions they ask. They dwell on traditional return on investment: how much does the tool cost? How much in professional services to stand it up? How much will we spend on credits? Those aren’t wrong questions, but they measure the tech in isolation, without weighing the risk or the opportunity.
Most RFPs still measure features and integrations. They don’t measure whether the platform can reason about your environment, learn from your analysts, or act autonomously across the threat lifecycle. Those three capabilities are the whole ballgame in an AI SOC, and the standard scorecard doesn’t even have a column for them.
The real question isn’t “How many features does it have?” It’s whether you can withstand the first wave of a machine-speed attack at all.
Start With a Trajectory, Not a Shortlist
If you take one practical answer from this piece on how to evaluate an AI SOC, make it this one. Most evaluation failures I’ve seen start before any vendor enters the room. They start with the absence of a trajectory.
Every strong evaluation I ran as a CISO began the same way: a clear view of our current state, the outcomes we wanted to deliver, and the actions required to get there, all within our risk threshold.
Where I see CISOs go wrong today is trying to evaluate everything at once, without a defined path. Establish your trajectory first, and you’ll know quickly whether you’re even talking to the right vendors. (How to actually build that trajectory is one of the first things I walk through in the guide.)
Know What You’re Actually Buying
Here’s the shift that matters most: You’re not just buying a tool. You’re buying an execution layer that will reshape your team, your operating model, and the way your organization defends itself.
That’s a fundamentally different purchase, and it changes what you should be measuring. The guide breaks down the model I use to think about it, as well as the AI SOC org chart and where your analysts go.
For now, the point is simply this: Evaluate the purchase knowing it changes how your whole function operates, not just which console your analysts log into.
The Questions This Series Will Answer
Once you have your trajectory, you can hold every vendor to the same standard. When every vendor is claiming “agentic,” a handful of questions separate the real platforms from the marketing. Across this series, I’ll go deep on the ones I think matter most — starting with whether a platform can genuinely reason about your environment, whether it actually learns from your analysts, and how much you can trust it to act on its own.
Know these going in, and the marketing falls away fast. If a vendor can’t help you take your first steps or walk you through how to get started, you’re talking to the wrong vendor.
The Budget Cycle Is Coming
These are the things to be thinking about as you put next year’s plan together. The AI SOC is a different purchase, so evaluate it like one: start with your trajectory, understand you’re buying an execution layer and not a tool, and hold every vendor to the questions that actually determine whether the investment succeeds.
I pulled everything I wish I’d had on the buying side into one place. The guide is the long answer on how to evaluate an AI SOC, with the questions to bring into every vendor meeting.
