🛡️

Running AI agents in production? Harness governs spend, access, and audit trails—so your team maintains control while agents safely handle production workflows. Visit →

OWASP Top 10 for LLMs

Deploying Large Language Models (LLMs) in production introduces specific threat vectors that traditional web application security models (like the standard OWASP Top 10) do not fully cover.

This playbook details the key vulnerabilities in the OWASP Top 10 for LLM Applications and provides concrete implementation-level mitigations for software engineers.

🛡️ 1. Secure Agent Execution Architecture

When securing agentic workflows, always intercept model actions before they escape the application perimeter:

🚨 2. Complete OWASP Top 10 Playbook

LLM01: Prompt Injection

Attackers use direct or indirect prompts to override model instructions, leading to data leaks, tool hijacking, or compliance violations.

Mitigation: Implement strict XML tag boundaries, employ input classification models (e.g., Llama Guard), and treat all external fetched context as untrusted payload data.

LLM02: Insecure Output Handling

When system developers trust LLM outputs blindly, passing them directly to compilers, databases, or client browsers without validation. This results in Cross-Site Scripting (XSS), SQL Injection, or Remote Code Execution (RCE).

Mitigation: Encode and sanitize all LLM-generated text before displaying it in client UI views. Force structured schema parsing (using Pydantic) to strip out executable tokens.

LLM03: Training Data Poisoning

Malicious manipulation of pre-training or fine-tuning datasets, causing the model to generate backdoors, biased outputs, or safety violations.

Mitigation: Audit and verify data supply lines. Use content hashing for training records and implement strict data purification pipelines to clean datasets before training.

LLM04: Model Denial of Service (DoS)

Flooding LLMs with resource-intensive requests (e.g., extremely long contexts or infinite recursion loops), causing service degradation, high latency, or financial exhaustion.

Mitigation: Implement strict token limits, set execution timeouts, monitor active execution loops, and apply rate limiting based on client request sizes and counts.

LLM05: Supply Chain Vulnerabilities

Relying on compromised third-party base models, fine-tuning datasets, Python packages, or plugins, which introduces hidden backdoors or security flaws.

Mitigation: Verify model hashes and authors. Pin all dependency versions, scan third-party packages for vulnerabilities, and use trusted, vetted open-source model registries.

LLM06: Sensitive Information Disclosure

The model reveals proprietary system instructions, database schemas, API keys, or private user data stored in its weights or context window.

Mitigation: Implement strict input sanitizers (scrubbing PII in queries) and output redact filters (regex checking response blocks for phone numbers, API keys, or credit cards).

LLM07: Insecure Plugin Design

Plugins called by LLMs accepting unvalidated inputs or lacking authorization checks, allowing attackers to exploit excessive privileges (e.g., executing arbitrary SQL via a database plugin).

Mitigation: Treat all inputs to plugins as untrusted payload data. Enforce strict JSON schema validation, use parameterized calls, and require user authentication on the plugin endpoint itself.

LLM08: Excessive Agency

An agent is granted too much authority, unsafe tools (like direct CLI execution), or lacks confirmation gates, allowing it to perform catastrophic actions (e.g., deleting servers or sending spam) when hijacked.

Mitigation: Apply least-privilege scoping to tools, enforce human-in-the-loop (HITL) gates on critical state changes, and run execution nodes inside sandboxed environments.

LLM09: Overreliance

Blindly trusting model outputs without verification, leading to the ingestion of incorrect code, security hallucinations, or false information into production systems.

Mitigation: Implement automated test suites for model-generated code, use secondary evaluator LLMs for factuality cross-checks, and ensure clear user warnings regarding model-generated advice.

LLM10: Model Theft

Unauthorized exfiltration of model weights, proprietary system prompts, or configuration parameters through side-channel attacks, prompt extraction, or direct API harvesting.

Mitigation: Rate limit API access, use prompt obfuscation techniques, implement egress watermarking, and protect model weights with strict enterprise storage access control lists.

💻 3. Defensive Code Implementation

The python script below illustrates how to mitigate Excessive Agency and Insecure Output Handling when an LLM requests code execution.

We restrict imports, limit runtime duration, validate parameters, and isolate execution inside a thread boundary:

import sys
import multiprocessing
import traceback
from typing import Dict, Any
 
def execute_untrusted_python_code(code_string: str, timeout_seconds: float = 2.0) -> Dict[str, Any]:
    """
    Executes an untrusted code string in a sandboxed subprocess boundary,
    preventing direct shell access and infinite execution loops.
    """
    def worker(code: str, queue: multiprocessing.Queue):
        # 1. Block dangerous built-in modules
        dangerous_imports = ["os", "sys", "subprocess", "shutil", "socket", "requests"]
        for module in dangerous_imports:
            sys.modules[module] = None # Disable module import resolution
 
        local_vars = {}
        try:
            # 2. Execute code in restricted globals/locals space
            exec(code, {"__builtins__": __builtins__}, local_vars)
            queue.put({"status": "success", "result": local_vars.get("result")})
        except Exception as e:
            queue.put({"status": "error", "error": str(e), "trace": traceback.format_exc()})
 
    queue = multiprocessing.Queue()
    process = multiprocessing.Process(target=worker, args=(code_string, queue))
    process.start()
 
    # 3. Enforce maximum execution duration (Mitigates Denial of Service loops)
    process.join(timeout=timeout_seconds)
    
    if process.is_alive():
        process.terminate()
        process.join()
        return {"status": "blocked", "reason": f"Execution timed out after {timeout_seconds} seconds."}
 
    if queue.empty():
        return {"status": "error", "reason": "No response returned from execution sandboxed thread."}
        
    return queue.get()
 
# Example usage:
# safe_code = "result = sum([1, 2, 3])"
# print(execute_untrusted_python_code(safe_code)) # Returns {'status': 'success', 'result': 6}
 
# malicious_code = "import os; os.system('rm -rf /')"
# print(execute_untrusted_python_code(malicious_code)) # Returns {'status': 'error', ...}

Agent Security & Guardrails — Comprehensive walkthrough of sandbox virtualization technologies (WASM, Firecracker, Docker) and network isolation.
Prompt Security & Defenses — Defending against jailbreaks, leaks, and indirect context injections.
Access Control & Authorization — Row-level security for vector databases and policy validation engines.

🔐 Access Control and Authorization 🚦 Safety Guardrails and Compliance