Basic Prompting & Primitives
Before implementing complex agent loops or advanced reasoning structures, developers must understand the foundational primitives of prompt engineering. Structuring prompts correctly ensures predictable outputs, prevents instruction drifting, and optimizes API transaction costs.
🎭 1. Message Roles and Scopes
Foundation chat APIs (OpenAI, Anthropic, Gemini) structure interactions into a conversation list where each message has a defined Role and Scope. Enforcing this separation prevents the model from conflating instructions with user data.
┌────────────────────────────────────────────────────────┐
│ System Message │
│ "You are a database analyst. Output answers as JSON." │
└──────────────────────────┬─────────────────────────────┘
▼
┌──────────────────────────┴─────────────────────────────┐
│ User Message │
│ "Analyze sales logs: <data>...</data>" │
└──────────────────────────┬─────────────────────────────┘
▼
┌──────────────────────────┴─────────────────────────────┐
│ Assistant Message │
│ "{"sales": 10240, "trends": "positive"}" │
└────────────────────────────────────────────────────────┘System Message
- Purpose: Configures the persona, behavioral boundaries, default capabilities, and response constraints of the model.
- Authority: The system prompt sits at the highest layer of instruction priority. It defines the global rules that the model must follow throughout the lifecycle of the session.
User Message
- Purpose: Contains the specific query, unstructured input data, or dynamic parameters provided at runtime.
- Authority: Subordinate to the system instructions. User inputs must be treated by the system instructions as untrusted data to prevent prompt injections.
Assistant Message
- Purpose: Represents the model’s own historical outputs.
- Authority: Used to maintain conversation memory. When managing multi-turn chats, appending assistant messages simulates episodic state history.
📐 2. Prompt Layout Engineering
To prevent the model from getting confused when processing large documents or multiple parameters, use structured markup to establish clear layouts.
XML Delimiters (The Industry Standard)
Modern LLMs (particularly Claude and Gemini) are trained to recognize XML tags. Wrapping user data, contexts, and instructions inside tags allows the model to cleanly isolate variables.
Act as a document translator. Translate the text enclosed in the <translate_text> tags.
Do not translate terms contained in the <glossary> tags.
<glossary>
- Vector Store -> Base de Vectores
- Reranking -> Re-clasificación
</glossary>
<translate_text>
The search engine utilizes a Vector Store for initial retrieval and a Cross-Encoder for Reranking.
</translate_text>Structured Output Framing
Relying on natural language requests like “Output in JSON” often leads to formatting failures (e.g., the model writing conversational prefixes like “Here is your JSON:”). To guarantee parseable outputs:
- Define Structured Schemas: Provide explicit JSON schemas or Pydantic definitions.
- Use Target Prefixes: End the final user message with the opening brace
\{(if supported by the API) to pre-fill the assistant response and force immediate JSON output. - Structure Few-Shot Examples: Match the structure of target formatting output in the prompt context.
✂️ 3. Token Budgeting, Caching & Context Engineering
Every prompt token sent to an LLM incurs financial cost and adds processing latency. Active prompt budgeting and context engineering are critical for production-grade scaling.
Context Engineering and Long-Context Patterns (Lost in the Middle)
In models supporting 1M+ token context windows (e.g. Claude 3.5 Sonnet, Gemini 2.0 Pro), retrieval performance is not uniform. The model exhibits a U-shaped recall curve—a phenomenon known as Lost in the Middle:
Recall Recall Accuracy (%)
▲
100 | ██████ ██████
| ██████ ██████
| ██████ ██████
| ██████ ██████ ██████ ██████
| ██████ ██████ ██████ ██████ ██████ ██████
0 └─────────────────────────────────────────────────────────►
0% (Start) 50% 100% (End)
Context Window Position- Rule of Margins: Place your critical reference facts, documents, or data payloads at the absolute beginning of the prompt window. Place your task instructions and output format constraints at the absolute end.
- XML Tag Navigation: Anchor document segments with clear tags (e.g.
<document id="doc_1">...</document>). When asking questions, reference these IDs directly so the model’s attention mechanism can instantly target the correct coordinate index.
Prompt Caching and the “Stable Prefix” Rule
Provider prompt caching (e.g. Anthropic Prompt Caching, Google Gemini Context Caching) reduces API costs by up to 50% and latency by up to 80% for static text payloads. However, to trigger caching, you must adhere to the Stable Prefix constraint:
- Sequential Caching: Caches are read and written sequentially from the beginning of the prompt.
- Dynamic Variable Placement: Any dynamic variable (e.g. user query, current timestamp, session token, or random seed) placed at the beginning of the prompt will invalidate the cache for all subsequent text blocks.
- Optimal Structure:
1. System Instructions (Static - Cached) 2. Reference Documentation/Data (Static - Cached) 3. User Query & Session Variables (Dynamic - Placed at the very end)
Context Truncation
Implement active context cleaners. Rather than passing entire chat histories, summarize older turns or prune system contexts that are no longer relevant to the current task. Avoid bloated system instructions; keep instructions targeted to the immediate task, and route complex flows into multi-step pipelines rather than a single massive “Swiss Army Knife” prompt.
🔗 Related Sections
- Prompting Techniques — Advanced reasoning templates like CoT, ToT, and ReAct loops.
- Prompt Hacking — Direct/Indirect injections and prompt protection templates.
- Agent Memory Systems — Dynamic context pruning and chat history summaries.