Hermes: Learning Systems & Trajectory Evolution

In the engineering of autonomous agents, a major bottleneck is the static nature of tool schemas. If an agent is limited to developer-defined functions compiled at startup, it cannot adapt to unique execution environments or correct its own operational failures.

This page details the architecture of Learning Systems, focusing on how persistent agents evaluate their own execution trajectories, author new code skills, and scale their tool registries.

🛠️ Why Static Tools Stop Improving

Traditional tool calling relies on static registries (e.g. JSON schemas declared in Python code). While this provides stability, it introduces key engineering limitations:

• Compile-time Boundaries: The agent cannot solve tasks that require a sequence of commands not explicitly pre-programmed.
• Context Bloat: Loading all possible tool definitions into the prompt exhausts the LLM’s context window and degrades retrieval accuracy.
• No Self-Correction: When a tool fails due to an environmental change (e.g., a changed database schema), a static agent cannot rewrite the tool logic to fix the error.

📐 Architecture: Engineering Persistent Learning Systems

Persistent learning systems solve these limitations by treating tools as dynamic assets stored in a local workspace. The system operates on a continuous feedback loop:

  [ Ingress Prompt ]
          │
          ▼
   ┌───────────────┐
   │ Prompt Builder│ <─── [ Vector Search: Match Semantic Skills ]
   └──────┬────────┘
          │
          ▼
   ┌───────────────┐
   │   Provider    │ ───> [ LLM Core Reasoning ]
   │  Resolution   │ <─── [ Return Trajectory Steps ]
   └──────┬────────┘
          │
          ▼
   ┌───────────────┐
   │ Tool Dispatch │ ───> [ Execute Code / Track Outcomes ]
   └───────────────┘

Rather than carrying a massive hardcoded list of APIs, the agent utilizes a vector database to search for, retrieve, and load only the tool definitions relevant to the current query (Progressive Disclosure).

📂 The Skill Lifecycle: Creation, Compilation, and Registry Storage

Dynamic skills pass through four key phases in their lifecycle:

1. Discovery: The agent encounters a task failure or a novel execution sequence.
2. Compilation: The agent drafts, tests, and validates a clean Python or bash script to resolve the task, formatting the schema under the agentskills.io standard.
3. Registry Injection: The compiled skill is saved as a markdown asset in the local /skills directory and indexed in the vector database.
4. On-Demand Loading: When a similar prompt is parsed, the system embedding model matches the query, pulls the skill metadata, and injects it into the prompt.

🤖 Case Study: Nous Hermes Agent

Developed by Nous Research, the Hermes Agent serves as a prime reference implementation of a persistent learning system. It runs locally as a background process and uses a custom workspace configuration containing a MEMORY.md file (for semantic context) and a /skills directory. The agent continuously monitors its own CLI executions, and whenever it successfully executes a complex code operation, it automatically compiles the path into an agentskills.io-compliant skill document.

💻 Code Playbook: Trajectory Evaluator & Skill Compiler

Below is a Python implementation of a Trajectory Evaluator. It simulates an agent grading its execution telemetry, checking for semantic duplication, and compiling a new skill:

import json
import logging
from typing import Dict, List, Any, Tuple
 
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("HermesPipeline")
 
class TrajectoryEvaluator:
    def __init__(self, existing_skills: List[Dict[str, Any]], score_threshold: float = 0.85):
        self.existing_skills = existing_skills
        self.score_threshold = score_threshold
 
    def evaluate_trajectory(self, trajectory: Dict[str, Any]) -> Tuple[bool, str]:
        """
        Grades a task execution trajectory and determines if it qualifies
        to be compiled into a reusable skill.
        """
        task_name = trajectory.get("task", "unknown_task")
        actions = trajectory.get("actions", [])
        success = trajectory.get("success", False)
        
        # Calculate execution performance metrics
        latency = trajectory.get("latency_seconds", 0.0)
        token_count = trajectory.get("tokens_consumed", 0)
        
        if not success:
            return False, "Skipping: Trajectory resulted in execution failure."
 
        # Compute performance score (penalizing high latency and tokens)
        base_score = 1.0
        latency_penalty = min(0.3, latency * 0.01)  # Max 0.3 penalty
        token_penalty = min(0.3, (token_count / 8192) * 0.1)
        score = base_score - latency_penalty - token_penalty
        
        logger.info(f"Trajectory evaluated for '{task_name}'. Performance Score: {score:.2f}")
 
        if score < self.score_threshold:
            return False, f"Skipping: Score ({score:.2f}) below threshold ({self.score_threshold})."
 
        # Check semantic duplicate overlap against existing tool signatures
        for skill in self.existing_skills:
            if skill["task_signature"] == task_name:
                overlap_ratio = self._calculate_action_overlap(actions, skill["actions"])
                if overlap_ratio > 0.8:
                    return False, f"Skipping: Action patterns overlap ({overlap_ratio * 100:.1f}%) with '{skill['name']}'."
 
        return True, "Passed: Trajectory qualifies for skill compilation."
 
    def _calculate_action_overlap(self, actions_a: List[str], actions_b: List[str]) -> float:
        set_a, set_b = set(actions_a), set(actions_b)
        intersection = set_a.intersection(set_b)
        union = set_a.union(set_b)
        return len(intersection) / len(union) if union else 0.0
 
class SkillCompiler:
    @staticmethod
    def compile_skill(trajectory: Dict[str, Any]) -> Dict[str, Any]:
        """
        Transforms the successful trajectory into an agentskills.io compliant document.
        """
        task = trajectory["task"]
        actions = trajectory["actions"]
        logger.info(f"Compiling new skill for task: {task}...")
        
        # Synthesize skill document
        skill_doc = {
            "name": f"auto_{task.replace(' ', '_').lower()}_skill",
            "task_signature": task,
            "actions": actions,
            "agentskills_version": "1.0.0",
            "doc_body": f"""---
name: "auto_{task.replace(' ', '_').lower()}_skill"
task_signature: "{task}"
tools_required: {actions}
version: "1.0.0"
---
 
# Skill: {task}
 
## System Instructions
Always execute the following tool sequence:
{chr(10).join([f'{i+1}. {act}' for i, act in enumerate(actions)])}
"""
        }
        return skill_doc
 
# Mock run
active_library = [
    {
        "name": "auto_ssh_ping_skill",
        "task_signature": "Ping Servers",
        "actions": ["ssh_connect", "send_ping"]
    }
]
 
mock_trajectory = {
    "task": "Extract Postgres Logs",
    "actions": ["ssh_connect", "grep_search_errors", "scp_transfer"],
    "success": True,
    "latency_seconds": 1.2,
    "tokens_consumed": 1200
}
 
evaluator = TrajectoryEvaluator(existing_skills=active_library)
qualifies, reason = evaluator.evaluate_trajectory(mock_trajectory)
logger.info(f"Evaluation Decision: {reason}")
 
if qualifies:
    new_skill = SkillCompiler.compile_skill(mock_trajectory)
    logger.info(f"Compiled Skill Document:\n{new_skill['doc_body']}")

💻 Code Playbook: Trajectory JSON Export Schema

Once an execution trace succeeds, it is formatted and exported. These JSON trajectory logs are used to fine-tune future LLM weights, aligning the model directly on optimal tool-use patterns:

{
  "trajectory_id": "traj_984f7e2a_2026",
  "task": "Extract Postgres Logs",
  "model_signature": "hermes-3-llama-3.1-70b",
  "performance": {
    "success": true,
    "score": 0.94,
    "latency_seconds": 1.2,
    "tokens_consumed": 1200
  },
  "steps": [
    {
      "step": 1,
      "thought": "Establish a secure connection to the remote database server.",
      "tool_call": {
        "name": "ssh_connect",
        "arguments": {
          "host": "db.internal.net",
          "user": "root"
        }
      },
      "tool_output": {
        "status": "success",
        "message": "Connected to db.internal.net (SSHv2)"
      }
    },
    {
      "step": 2,
      "thought": "Locate log entries matching 'CRITICAL' in the Postgres directory.",
      "tool_call": {
        "name": "grep_search_errors",
        "arguments": {
          "pattern": "CRITICAL",
          "filepath": "/var/log/postgresql/postgresql-15-main.log"
        }
      },
      "tool_output": {
        "status": "success",
        "match_count": 42,
        "filepath": "/tmp/grep_output_postgresql-15-main.log"
      }
    }
  ]
}

📈 Dynamic Skill Optimization & Scaling

Always-on learning agents require strict registry controls to prevent degradation over time:

• Skill Pruning: Establish a threshold for usage frequency. If an agent-generated skill has not been called in 100 sessions, archive it out of the vector database index.
• Vector Registry Scaling: As the registry grows to thousands of skills, standard cosine similarity search can return noisy matches. Implement Hierarchical Navigable Small World (HNSW) indexing with category-based metadata filters (e.g. filter by category: database before querying).
• Token Budget Management: Limit the system prompt injection to a maximum of the top 3 semantically matched skills. Adding more than 3 schemas increases token costs and runs the risk of LLM attention-loss (needle-in-a-haystack dropoff).
• Memory Growth & Maintenance: Periodically run a summarizer agent over MEMORY.md logs to consolidate repeating traces into consolidated behavioral summaries, preventing the context file from growing infinitely.

⚖️ Engineering Decisions & Trade-offs

When designing a dynamic learning registry, consider the following architecture guidelines:

Design Trade-offs

• Dynamic Compilation vs. Prompt Size: Dynamically writing skills reduces initial prompt size and context bloat, but increases execution latency because the system must run evaluations and vector database lookups before starting tasks.
• Safety vs. Autonomy: Granting an agent permission to inject its own code scripts into the active tool dispatcher increases capability, but creates a high risk of loop execution loops if a buggy script is compiled.

Production Best Practices

• Execute inside a Sandboxed Environment: Never run agent-compiled code on the host machine. Ensure the skill execution engine points to an isolated container.
• Implement Human-in-the-loop Gates: For critical operations (e.g. database deletes or network requests), enforce an approval check before registering the newly compiled skill.

Decisive Guidance

• When to Use: Use dynamic learning systems when the agent operates in complex, unstructured environments where the exact sequence of tools cannot be predicted on compile-time (e.g. custom web automation and multi-repo code maintenance).
• When to Avoid: Avoid in deterministic application pipelines where predictability, latency bounds, and strict security compliance are the primary objectives (e.g., banking transactions or customer support routing).