How App Integration Transactions Increase the Attack Surface of LLMs

From a security perspective, evolving our approach is critical. We’ll soon be facing an increase in scenarios where compromised and/or manipulated LLMs can interact with entire ecosystems of commerce and service platforms. This will require LLMs to be more restrictive and security-oriented when filtering prompt injections.   

The bottom line: this isn’t just another feature update. It’s a fundamental expansion of what LLMs can do, making it easier to access functionality that was technically possible before, but required more effort and understanding of the process for integration. Now, with less thought into what’s happening behind the scenes and how these agent ensembles are working together, our security measures need to evolve accordingly. 

The Weakest Links in the AI Chain 

The security of these new transactional AI systems depends on multiple factors, many of which are prone to failure, so much so that through 2029, over 50% of successful cybersecurity attacks against AI agents will exploit access control issues, using direct or indirect prompt injection as an attack vector. Most companies rely on one of the five mainstream model providers: OpenAI, Google, Meta, xAI or Anthropic. This consolidation means that businesses are at the mercy of the provider’s implementation quality and the model’s hallucination rate. Granted, those are generally more resilient than a self-hosted and trained model. 

However, the most significant vulnerabilities often arise from third-party integrations, as these introduce more moving parts and variability. Common issues include: 

• Poor Authentication/Authorization: Inadequate verification of user identity and permissions can allow attackers to impersonate users or escalate privileges. 
• Weak System Prompt Controls: Insufficiently robust system prompts that define the AI’s behavior and limitations can be bypassed, allowing attackers to override intended safeguards via direct and/or indirect prompt injection. 
• Race Conditions and Workflow Integrations: The complex interplay between the LLM, third-party apps, and user inputs can create race conditions or other workflow vulnerabilities that attackers can exploit to bypass security checks. 

Why “Buy It” Is Just the Beginning 

OpenAI is the first to pioneer this type of in-house feature for their LLM, but others will likely follow suit. The reality is that LLMs are currently extremely expensive. Training and operating these models are incredibly resource-intensive, with reports indicating OpenAI’s costs ran into the billions in 2024. As a result, AI model providers will continue pursuing new revenue streams. Introducing features, ad revenue, and transaction fees from e-commerce integrations seems like the next logical step. 

As this technology matures, more applications will adopt it, expanding the attack surface and increasing the urgency for thorough security testing and validation with partners like NetSPI. 

Secure LLM Implementations with NetSPI