Gal Peretz is the former Head of AI & Data at Torq. Gal accelerates Torq’s AI and data initiatives, applying his deep learning and natural language processing expertise to advance AI-powered security automation. He also co-hosts the LangTalks podcast, which discusses the latest AI and LLM technologies.

Our previous blog post explored how planning with AI systems can set the stage for smooth collaboration between humans and machines. However, a solid plan alone isn’t enough. The next step is orchestrating the execution — ensuring that the AI system can carry out tasks autonomously while maintaining guardrails that prevent errors, hallucinations, or false actions.

The Challenge of Direct Execution: LLMs Alone Aren’t Enough

Moving from a free-text runbook to execution without a structured schema is where most AI implementations fail. While large language models (LLMs) are powerful, they continue to struggle with AI autonomy due to:

• Hallucinations: Making incorrect assumptions or executing invalid steps.
• Ambiguity: Choosing the wrong tools or extracting incorrect arguments from the execution context to pass to the next step.
• Lack of Determinism: Struggling to execute tasks consistently without a clear structure, often leading to indeterministic execution where the AI agent may jump between steps out of order or skip them altogether.

Simply put, letting the LLM orchestrate execution without structure and guardrails lacks the precision needed for a reliable execution process.

Streamlining the Execution of a Clear and Reliable Plan

To address this, Torq implements a concrete structured execution scheme that ensures Torq’s AI system performs tasks deterministically and without ambiguity. Once the high-level plan is developed, the AI extracts each step as an atomic unit — clear, precise, and sequential. 

This structured approach eliminates the risks of indeterministic execution, where the AI agent might skip steps, go out of order, select incorrect tools, or misinterpret arguments due to vague instructions. 

Think of it like following a recipe step by step: after deciding to ‘make dinner,’ you break your activities into clear, sequential micro-tasks like ‘bring water to a boil’, then ‘add pasta to the water.’ 

Similarly, Torq built our AI to execute a detailed plan one micro task at a time, in the right order. This allows Torq’s AI system to analyze and break down instructions and examples for each step, ensuring the AI completes the overarching task accurately. By eliminating ambiguity, the structured execution guides the AI to select the right tools and arguments at every stage, delivering consistent and reliable results.

AI Guardrails: Balancing AI Autonomy and Control

While we aim to maximize AI autonomy, balancing it with guardrails is critical to ensuring its safe and reliable execution. These guardrails act as safety nets that prevent the AI from taking false or unintended actions, ensuring human oversight remains available when necessary.

The key is for the AI to be able to break down the execution process into atomic steps that it can handle precisely. The system then focuses on clear micro-tasks for each step, reducing ambiguity and enabling the AI to perform confidently. 

However, when the AI encounters uncertainty — such as ambiguous context, missing tools, or incomplete arguments — it pauses execution and escalates the decision to a human operator. This human-in-the-loop mechanism mitigates the risks of hallucinations or incorrect tool usage, providing a safety checkpoint before the AI proceeds.

By combining structured execution with these dynamic guardrails, we can push the boundaries of AI autonomy. This allows the AI to operate efficiently and autonomously in most cases, saving significant time and resources while ensuring that safety and accuracy are never compromised.

Reliable AI-Powered Execution at Scale

Orchestrated execution unlocks AI’s full potential by combining precision, autonomy, and control. By leveraging a step-by-step structure, AI can focus on atomic tasks, ensuring consistency and reliability at every stage. This approach streamlines workflows requiring constant human intervention, enabling AI to act efficiently while remaining grounded in a structured plan.

For Security Operations Center (SOC) teams, this translates to faster and more reliable execution of security runbooks at scale. This reduces the need to micromanage AI-powered SOC processes or perfect the prompts to control the AI, giving SOC teams more time for higher-value tasks while ensuring confidence in the AI’s structured execution.

The Future of AI Autonomy in the SOC

Choosing solutions that orchestrate AI execution with appropriate guardrails is critical for building trust, efficiency, and precision in today’s SOC operations. AI that structures execution as a series of deterministic micro steps and balances AI autonomy with human oversight allows SOC teams to confidently rely on AI systems to streamline their workflows.

This collaborative approach enables SOC analysts, engineers, and managers to:

• Maintain control over automated processes
• Trust in AI’s reliability for step-by-step execution
• Focus on higher-value work while reducing uncertainty

The result is a stronger, more efficient autonomous SOC environment where human expertise and AI capabilities work seamlessly together. Schedule a demo today.

As C-suites and boards are bombarded with headlines about AI revolutionizing cybersecurity, it’s no wonder they’re putting pressure on SOC leaders to adopt AI. The promise of AI in the SOC is rightfully alluring. An AI-native autonomous SOC has the potential to create a world where AI Agents collaborate with each other to take care of repetitive tasks and handle the majority of low-level alerts, freeing your human team up for strategic, proactive work. 

The hurdle? The AI cybersecurity landscape is swarming with vendors — and new ones are seemingly popping up out of stealth mode every day with shiny marketing and grand claims. 

This leaves SOC leaders wading through the noise to figure out which tools are overexaggerated AI-washed vaporware and which ones are truly operational, integrated, and trustworthy. Below are some tips for cutting through the hype to find the right AI solutions to help build an autonomous SOC. 

Start with the End Goal in Mind — and Think Big Picture

Step back and start with the big picture. To avoid “scattergun” AI adoption in the SOC that leads to a flood of AI-generated alerts with no context or prioritization, begin by defining clear AI objectives aligned with your overarching security strategy. Before you dive into the AI vendor pool, take a moment to reflect on your SOC’s practical needs. What are your biggest pain points? Where could AI make the biggest impact? Are your analysts drowning in a sea of alerts? Or are they having to spend too much time on tedious tasks? Prioritize AI solutions that directly address these day-to-day challenges.

“I believe the successful use of AI in SOC operations shows up in practical outcomes. With Torq Agentic AI, the answer is ‘yes’ to questions such as: Are analysts happier? Are they sticking around? Do they have time to focus on more interesting and complex investigations? Are MTTM and MTTR lower? Torq Agentic AI extends and enhances our team so it can make better decisions more quickly — resulting in stronger security all around.”

– Mick Leach, Field CISO, Abnormal Security

Leverage AI for tasks where human limitations — such as fatigue and information overload — lead to inefficiencies. Generative AI-powered AI Agents are adept at tasks involving natural language processing and the creation of logical workflows. This makes AI ideal for automating repetitive, monotonous tasks, intelligently triaging alerts and autonomously handling incidents, and providing real-time insights and recommended next steps to speed up human decision-making. In turn, human analysts are freed up to focus on strategic activities and make faster, more informed decisions, significantly improving overall efficiency and effectiveness.

Think holistically to maximize the value of your investment. One-off AI tools from different vendors can’t add up to an autonomous SOC because they can’t connect security signals across your stack and provide meaningful, context-rich insights. Prioritize investing in a centralized automation platform with enterprise-grade scalability and the ability to integrate with every solution in your security environment. Purpose-built AI Agents for the SOC built on this foundation can work as a unifying force at the heart of your security stack to correlate disparate event data, uncover deep, contextual insights, and accelerate efficiency gains across your security operations.

Stay ahead of threats by keeping up with autonomous SOC advancements. Hyperautomation is now table stakes for Security Operations, demanding platforms with native, fully embedded AI capabilities rather than bolted-on GPT wrappers. Agentic AI,  the new frontier for delivering on the promise of the autonomous SOC, is now a reality. Torq just announced a groundbreaking Multi-Agent System for security operations with specialized AI Agents that collaborate, plan, and reason to autonomously analyze and resolve security threats. 

“SecOps organizations that adopt GenAI-based Hyperautomation will benefit from the most advanced LLMs ever, enabling analysts to auto-analyze more events and identify novel threats at the beginning of their cascade of potential impact, rather than after they’ve had a chance to create serious damage. GenAI will also further democratize SecOps, so employees at all levels are able to deploy, manage, and monitor Hyperautomation systems.”

– Leonid Belkind, Torq CTO and Co-Founder | 2025 Predictions: How GenAI and Hyperautomation Will Reshape SecOps and Threat Landscapes

Tips for Evaluating AI Cybersecurity Tools for the SOC

Establish your evaluation criteria: Given the potential risks associated with AI solutions, careful third-party risk management is crucial.  Collaborate with IT teams, business leaders, and legal to ensure alignment with company-wide AI usage policies. Below are some key considerations when choosing a vendor for AI in the SOC:

• Flexibility and integration: Make sure the AI solution you choose can easily integrate with your existing security stack and ingest and intelligently transform data in any format. A flexible platform that can adapt to your evolving needs is essential so you don’t get locked in. 
• Security and privacy: Any solution deployed in your SOC should meet enterprise-grade security standards and have tiers of controls to protect data confidentiality. 
• Transparency: One of the most crucial elements for building trust in AI is to ensure the model can explain why it made the decisions it made and how it came to the conclusions it did. 
• Human-AI collaboration: Effective AI Agents in the SOC facilitate a collaborative, back-and-forth relationship with the human analysts they work with, clearly communicating its capabilities and limitations. When encountering roadblocks, the AI should seek human input or validation.

Ask the right questions: Overexaggerated,  misleading, and outright false claims about AI capabilities are all too common. We’ve got a list of 40 questions to help you understand a vendor’s AI capabilities, integrations, and more, such as:

1. Is all customer data encrypted in transit? Is stored data encrypted on disk? Is data stored in vendor data centers or only in memory? 
2. What countermeasures does the solution have in place to prevent AI hallucinations?
3. Does the system keep immutable records of all inputs and outputs for AI-driven actions?
4. Does the solution have robust and versatile role-based access controls? 

Refine your shortlist: Use your evaluation criteria to narrow down your list of potential vendors. Consider factors like cost, features, and vendor reputation. Conduct thorough research and request demos from your shortlisted vendors. 

Test before you invest: The proof of whether an AI solution is vaporware or truly operational is in the POC. Ask for demos and conduct a proof-of-concept for a key use case to see the AI solution in action in a controlled environment. Pay attention to the scalability, ease of use, and overall performance. 

Consider long-term partnerships: Build strong relationships with vendors who can provide ongoing support and innovation. Ask about their AI product roadmap.

40 Questions to Ask AI SOC Vendors

To help you sharpen your evaluation of AI solutions for the SOC, we’ve put together this list of 40 critical questions to ask vendors. Cut through the noise of “AI-washed” marketing and dig into the AI’s operational and integration capabilities to ensure you get real value.

Gal Peretz is the former Head of AI & Data at Torq. Gal accelerates Torq’s AI and data initiatives, applying his deep learning and natural language processing expertise to advance AI-powered security automation. He also co-hosts the LangTalks podcast, which discusses the latest AI and LLM technologies.

To stay ahead of today’s threats, you must do more than keep pace — you need to equip your team with tools that enable smarter, faster responses. For SOC analysts, runbooks in case management systems are essential guides for handling security alerts step-by-step. The prospect of automating these runbooks with AI is enticing, promising to streamline daily operations and free up time for more critical tasks.

However, some are rightfully skeptical. They worry that AI automation could introduce unexpected issues without careful planning and collaboration, potentially hindering productivity and increasing risk. This blog explores how collaborating with AI during planning and setting AI guardrails can enhance predictability, transparency, and trust in AI automation.

The Importance of Runbooks in Security Operations

Runbooks are structured, step-by-step guides enabling SOC analysts to respond to security incidents consistently and accurately. They are particularly crucial for Tier 1 analysts, who often serve as the first line of defense against a high volume of alerts. 

These runbooks provide clear instructions for the following:

• Triaging alerts
• Investigating potential threats
• Determining when to escalate issues

By standardizing responses, runbooks reduce human error and ensure efficient handling of all incidents, even in high-pressure situations. Automating runbooks with AI presents an appealing option for scaling operations, accelerating repetitive tasks, and allowing analysts to focus on more complex, high-stakes cases.

The Need for AI Guardrails in Runbook Automation

While automating runbooks with AI is a game-changer, granting AI too much freedom can quickly backfire. Most runbooks are designed with human readers in mind, presenting step-by-step guides that make sense to analysts but can be confusing for AI. 

When left to interpret these text-based instructions independently, AI might:

• Misinterpret steps
• Make unexpected decisions
• Produce unintended results

AI can become unpredictable without a structured plan and human alignment, risking accuracy and eroding your team’s trust in automation. A collaborative planning phase to ensure AI guardrails is crucial as it provides SOC analysts visibility into how the AI “interprets” the runbook and plans to automate it. This transparency allows analysts to refine the AI’s approach, ensuring the plan aligns with real-world needs before execution begins.

Collaborative Planning: Aligning AI and Analysts

To understand the value of Torq’s approach to runbook automation, let’s consider a common SOC runbook for investigating phishing reports. Such runbooks guide analysts through tasks like checking attachments, analyzing email headers, and escalating incidents when certain conditions are met.

Automating these tasks with AI is more complex than simply running through the steps. Many runbooks are written for human understanding and involve actions that may be ambiguous or beyond direct AI capabilities. Torq’s plan-and-execute approach addresses this challenge by separating the process into distinct planning and execution phases, giving analysts more control and visibility over the AI’s actions.

1. Planning Phase

In this phase, the AI:

1. Reads through the runbook
2. Converts instructions into a structured, transparent plan
3. Break down each instruction into clear, atomic steps
4. Identifies steps it can automate and those requiring human intervention
5. Highlights gaps where it lacks necessary tools or access

This transparency allows SOC analysts to modify the plan, choosing where the AI should pause for guidance or where additional human-defined workflows are needed. In scenarios where full automation isn’t feasible, such as in highly secure or restricted environments, this collaborative planning ensures that the AI aligns closely with human intent and avoids unnecessary errors.

2. Execution Phase

Once the analyst reviews and approves the plan, execution follows this carefully vetted blueprint. 

This approach:

• Strips ambiguity and indeterminism from the execution
• Provides transparency and reliability
• Fosters trust in the automation process

Analysts can be confident that AI will follow the exact plan, making the automation more efficient and dependable without sacrificing control or accuracy.

To reinforce the concept further, let’s consider how Socrates, our AI SOC analyst, would function without the ability to add tags while focusing on his communication skills and resistance to AI hallucination.

Socrates, even without the capability to add tags, would still demonstrate its effectiveness in several ways:

Clear communication of limitations: When faced with a task it cannot perform, such as adding a tag, Socrates would explicitly state its limitations. For example, it might say, “I’m unable to add the tag ‘Malicious IOC’ as I don’t have that capability. This step requires human intervention.”

Requesting user input: Socrates pauses the process and asks for user input when the necessary tools or permissions are lacking. This demonstrates its ability to recognize boundaries and seek assistance when needed.

Proceeding with available tools: For steps where Socrates has the required capabilities, it would continue to execute them efficiently. These actions would be marked as completed or “green” in the process.

Detailed explanations: Throughout its analysis and decision-making process, Socrates provides clear, thorough explanations of his reasoning, helping analysts understand its thought process even when it couldn’t perform specific actions.

Suggesting alternatives: When unable to perform a specific action, Socrates might suggest alternative approaches or provide information that could help the human analyst complete the task manually.

Focusing on these aspects can highlight Socrates’ ability to communicate effectively, recognize its own limitations, and resist AI hallucination by not claiming capabilities it doesn’t have. This approach emphasizes AI’s role as a collaborative tool that enhances human decision-making in the SOC rather than attempting to replace human judgment entirely. See what this looks like below:

Strengthening Security Through Transparent AI Collaboration

Trust and transparency are fundamental to building an effective security strategy in today’s rapidly evolving threat landscape. Torq’s AI capabilities prioritize collaboration and clarity, transforming how SOC teams handle automation. By structuring automation as a two-phase process — planning and execution — Torq ensures that AI usage is efficient, bounded by AI guardrails, and aligned with human oversight and intent.

This collaborative approach allows human SOC analysts to:

• Maintain control over automated processes
• Reduce uncertainty in AI actions
• Trust in the predictability and reliability of AI-driven tasks

Fostering a security environment where AI and human expertise work together can strengthen organizations’ SOC capabilities and enhance overall security posture. See Torq’s AI in action — schedule a demo.

Learn more about building trust in AI and how structured, evidence-backed summaries generated by AI enable seamless SOC shift transfers.