🛡️

Running AI agents in production? Harness governs spend, access, and audit trails—so your team maintains control while agents safely handle production workflows. Visit →

Safety Guardrails & Compliance

Deploying AI systems in enterprise environments requires structured moderation guardrails, aligned foundation models, and compliance audits. This guide details how to implement in-context classifiers, understand model-level alignment training, and align system development with international regulatory compliance frameworks.

🚦 1. In-Context Safety Classifiers

A safety classifier evaluates user queries and model generations against a safety policy before forwarding the tokens.

Llama Guard Moderation Pipeline (Python)

Llama Guard is a specialized LLM fine-tuned to classify queries into safety categories (e.g., violence, self-harm, cyberattacks, sexual content). Below is a Python implementation routing user inputs through a Llama Guard endpoint:

from openai import OpenAI
import os
 
client = OpenAI(
    base_url=os.getenv("LLAMA_GUARD_API_URL", "https://api.together.xyz/v1"),
    api_key=os.getenv("LLAMA_GUARD_API_KEY")
)
 
def moderate_query(user_query: str) -> bool:
    """
    Evaluates a user query against the Llama Guard safety policy.
    Returns True if the query is SAFE, and False if it is UNSAFE.
    """
    # Llama Guard expects system formatting defining the safety categories
    prompt_template = f"""[INST] Task: Check if there is unsafe content in the agent message in the credit card context.
    
    <categories>
    O1: Violence and Hate Speech.
    O2: Sexual Content.
    O3: Cyberattacks and Malware (e.g., hacking instructions, software vulnerabilities).
    O4: PII Exposure and Identity Theft.
    </categories>
    
    User Query: {user_query}
    
    Provide your output strictly in this format:
    unsafe
    <category_id>
    or
    safe
    [/INST]"""
 
    response = client.chat.completions.create(
        model="meta-llama/Meta-Llama-Guard-3-8B",
        messages=[{"role": "user", "content": prompt_template}],
        temperature=0.0 # Force deterministic output classification
    )
    
    result = response.choices[0].message.content.strip().lower()
    return "safe" in result
 
# Example check:
# is_safe = moderate_query("How can I extract SQL injection payloads from this website?")
# print(f"Query Safety: {'Approved' if is_safe else 'Blocked'}")

🛡️ 2. NVIDIA NeMo Guardrails

NVIDIA NeMo Guardrails allows developers to enforce behavioral boundaries using Colang schemas. Colang maps user intents, executes specific action rails, and overrides responses when user queries drift into forbidden topics.

Canonical Guardrail Configuration (Colang)

The example below defines an input rail that blocks users from asking our virtual customer assistant about competitive product pricing:

# config.co
define user ask about competitors
  "What do you think about competitor company X?"
  "Is competitor Y cheaper than you?"
  "Why should I choose you over competitor Z?"
 
define flow competitor block
  user ask about competitors
  bot refuse competitor talk
 
define bot refuse competitor talk
  "I am authorized only to answer questions regarding our product features and support documentation. I cannot discuss or compare competitor pricing."

⚖️ 3. Model-Level Alignment Optimization

While guardrails act as external wrappers, models themselves are aligned during post-training using reinforcement learning and preference optimization.

Alignment Method	Mathematical Objective	Training Complexity	Latency / Cost Impact	Rationale
RLHF (Reinforcement Learning from Human Feedback)	Maximizes reward model scores modeled on human preference rankings.	High (Requires active PPO reinforcement loops and separate critic models).	None (Only affects model weights at offline training time).	Standard alignment method (e.g. GPT-4). Provides highly conversational safety alignment.
DPO (Direct Preference Optimization)	Directly optimizes the policy’s log-likelihood ratio on preferred vs. rejected outputs.	Low (Eliminates the separate reward model, reducing training to simple binary classification).	None	Modern, stable alignment method. Prevents unstable reinforcement loops while keeping weights aligned.
KTO (Kahneman-Tversky Optimization)	Maximizes utility utility based on loss aversion metrics (describes how humans perceive utility).	Lowest (Does not require paired preferred/rejected datasets; evaluates single binary thumbs-up/down).	None	Best for real-time online learning, allowing systems to align models based on user telemetry feedback.

📋 4. Regulatory Compliance Frameworks

Enterprise AI systems must comply with global compliance mandates. When auditing your application, align your architecture to these controls:

EU AI Act Risk Tier Playbook

Prohibited Risk (Banned): Systems executing cognitive behavioral manipulation, untargeted facial scraping, or social scoring. (Forbidden from deployment).
High Risk: Systems used in critical infrastructure, medical devices, educational grading, or employment screening. (Requires mandatory human oversight, rigorous logging, and conformity assessments).
General Purpose AI (GPAI): Large foundation models (e.g. Claude, GPT). (Requires technical documentation, copyright compliance summaries, and systemic risk analysis for compute capabilities > 10^25 FLOPs).
Minimal Risk: AI-enabled spam filters, translation helpers, or search. (Minimal requirements, subject only to basic transparency rules like informing users they are interacting with AI).

NIST AI Risk Management Framework (RMF) Checklist

NIST Control Area	System Design Requirement	Engineering Verification
Govern	Establish clear ownership and tracking profiles for all models and training data sources.	Implement model registries with automated version metadata logging.
Map	Identify systemic vulnerabilities, potential biases, and prompt injection vectors.	Maintain threat modeling documentation and OWASP audit logs.
Measure	Quantify model accuracy, hallucination rates, and security boundary drift.	Run automated evaluation runs (Promptfoo) on CI/CD commits.
Manage	Enforce security sanitizers, active sandbox environments, and emergency rollback procedures.	Configure containerized tool execution environments and runtime guardrails.

Prompt Security & Defenses — Input sanitization techniques, Sandwich defense, and XML tag isolation.
Access Control & Authorization — Role-based and relationship-based access control strategies.
Agent Security & Guardrails — ephemaral sandboxes and network egress firewalls.

🛡️ OWASP Top 10 for LLMs ⚖️ Responsible AI & Ethics