���ϳԹ���

The Rise of Agentic AI: Is Your Cyber Team Ready?

Agentic AI is emerging as a transformative force, promising to redefine workflows across various industries, including cybersecurity. In our most recent webinar, The Rise of Agentic AI: Is Your Cyber Team Ready?, Immersive's Container 7 research team discussed the capabilities and risks associated with agentic AI and readiness imperative for cyber teams in this new era.

‍

Understanding Agentic AI: Beyond Generative Models

While many are familiar with generative AI and large language models (LLMs), agentic AI represents a significant step towards Artificial General Intelligence (AGI). AI agents are designed to break down complex tasks into smaller, manageable steps, allowing them to achieve more intricate objectives autonomously.

‍

The Double-Edged Sword: AI in Application Security and Offensive Operations

AI agents are increasingly being embedded into developer workflows, notably through tools like GitHub Copilot. While they can significantly boost efficiency, concerns arise when these agents propose installing packages or running commands automatically, without human oversight. There’s a clear danger in blindly trusting AI-generated code, just as there is in running unverified code from platforms like Stack Overflow.

Threat actors could also leverage agentic AI to accelerate attack lifecycles and streamline their processes in theory. These agents could enable the faster creation and deployment of attacks, as they would allow malicious actors to go from ideation to attack much more quickly.

‍

Critical Concerns: Trust, Data, and Human Oversight

Here are a few critical concerns to be aware of when it comes to agentic AI:

• Blind Trust: A significant risk is the prevailing attitude that "if it came from AI, it must be safe". This goes against fundamental security practices, such as verifying software bill of materials (SBOMs) and checking for vulnerabilities in AI-generated dependencies. AI, trained on historic data, may even favor outdated or vulnerable packages to reduce costs associated with fetching newer information.‍
• Data Handling and Compliance: Customer data isn't the pen-tester's to send to external AI providers without explicit consent. Agentic AI, in theory, could inadvertently exfiltrate sensitive customer data during a penetration test (e.g., PCI DSS compliant data) if it performs actions like a SQL dump and sends that data to a third-party AI processor. This poses significant GDPR and regulatory compliance risks to organizations, leaving them exposed to fines and other consequences.

• Ethical and Reputational Risks: Handing over control to AI agents, especially in risky scenarios, raises questions about accountability and liability if something goes wrong. Certain tasks just carry too high a risk of financial, regulatory, or reputational damage to be fully automated.
• Prompt Injection and Input Trust: When developers integrate AI into their products, they must treat AI inputs and outputs with the same caution as user inputs, verifying everything. Prompt injection remains a significant risk, especially as AI models are chained together and pass context between functions and other agents. Limiting the actions and access an LLM has is crucial.
• Business Logic Flaws: While AI might help fix simple vulnerabilities, it could introduce complex logic flaws that are harder to detect and can have catastrophic consequences. Regulators will hold the company, not the AI, accountable, for vulnerable code

The Human in the Loop: An Undeniable Imperative

The consensus is clear: people, processes, and businesses are not yet ready to fully hand over control to generative AI. While AI does enhance efficiency and innovation, humans must remain in the loop. With the emergence of tools like Googles Jules, designed to automate entire Git workflows, removing humans from every step of the development pipeline is cause for concern.

At a bare minimum, a human should always be present to check and approve the merge button to the main branch before deployment to production, essentially putting the irreputation on the line. Understanding the problem and defining clear boundaries for what AI should and absolutely should not do is paramount.

‍

Key Takeaways for Cyber Teams

As agentic AI becomes an undeniable part of the tech stack, cyber teams must:

• Trust, but Verify: Always verify what AI is doing and ensure it meets all security requirements.
• Understand and Be Proficient: Don't fear agentic AI, but prepare your teams to understand and be competent in using this technology responsibly.
• Strategic Deployment: Recognize that AI is not a "magic bullet." The more power given to these systems, the more damage can be done if not deployed with extreme care and clear boundaries.‍
• Adapt Security Practices: Incorporate testing of AI itself, understand how it changes the attack surface, and question the provenance of AI models.

The bottom line is Gen AI and AI agents are here to stay, and its potential is immense. However, its adoption requires a careful, responsible, and human-centric approach to ensure that efficiency gains do not come at the cost of security and accountability.

To learn more, access the full webinar recording .

‍