HermesAgent Memory Overview

Related: Hermes Memory Storage | Hermes Memory Retrieval | Hermes Memory Optimization | Hermes Context Management | Hermes Index

Memory is the part of the agent that persists information across turns and across runs. Without memory, every conversation starts cold; every task forgets the previous one. With memory, the agent accumulates context the way a human teammate does.

This note is the conceptual map. The next three notes cover storage, retrieval, and optimization in depth.

Why Memory Is a Separate Layer

A naive instinct is "the LLM has a context window — that's the memory." It isn't.

The context window is per-call. It's RAM. It's gone the moment the call ends. Memory is durable — it survives across calls, across sessions, across process restarts.

Splitting context (RAM) from memory (disk) gives you:

• Persistence across runs
• Selectivity — load only the relevant slice into context
• Scale — store gigabytes; load kilobytes per turn
• Multi-agent sharing — one memory; many agents see it
• Audit — query "what does the agent know about X?"

Engineering analogy: memory is the agent's filesystem. Context is what's in working RAM right now. The OS (the prompt builder) loads the relevant pages from disk as needed.

Three Memory Types

Short-term (working memory)

Lives in the prompt: the current transcript, recent observations, the current plan. Wiped at the end of a session.

You don't write code for short-term memory directly — it's the State object the orchestrator maintains. See Hermes Context Management.

Long-term (semantic memory)

Lives in storage: facts, summaries, knowledge. Survives across runs.

Examples:

• "User prefers tabular output over bullet lists"
• "Project X uses TypeScript, not Python"
• "Last week's report concluded the bottleneck was disk I/O"

Long-term memory is the main subject of this module.

Episodic memory

Lives in storage: complete past episodes (tasks, conversations, runs).

Examples:

• "On 2026-04-12, the agent successfully wrote report.md after 8 turns"
• "User asked about feature X on Apr 5 and we agreed to defer"

Episodic memory is replayable — you can ask the agent "what did we do last Tuesday?" and it can answer with citations to specific past runs.

These three are conceptually distinct, but a single backend can store them in different namespaces. See Hermes Memory Storage.

Anatomy of a Memory Entry

A memory entry is more than a string. It carries metadata that makes retrieval possible:

class MemoryEntry:
    id: str
    text: str                           # the content
    embedding: list[float]              # for semantic search
    metadata: dict                      # tags, source, timestamps
    created_at: datetime
    last_accessed: datetime
    access_count: int

The metadata is what lets you filter — "find memories tagged project-x from this month." See Hermes Memory Retrieval.

Memory Operations

Three core operations.

Write

The agent (or the orchestrator on its behalf) commits a fact:

memory.write(
    text="User's preferred report format is Markdown with H2 sections",
    tags=["preference", "format"],
    source="conversation_2026-05-02_session_42",
)

Writes happen:

• Explicitly via a remember() tool call
• Automatically when the orchestrator promotes a transcript fact (see Hermes Memory Optimization)
• At checkpoint time during long runs

Read (retrieve)

The prompt builder queries memory at the start of every turn:

relevant = memory.search(
    query=current_observation,
    k=5,
    filter={"tags": ["project-x"]},
)

Top-k results enter the prompt as the "Memory" section. See Hermes Memory Retrieval.

Forget (or decay)

Old, low-value memory should leave. Three approaches:

• Hard delete by id (rare; usually for privacy)
• Time-based decay — entries older than N months expire
• Access-based pruning — entries not retrieved in M months are dropped

See Hermes Memory Optimization.

Where Memory Lives in the Architecture

                ┌──────────────── AGENT ─────────────┐
                │                                    │
  observe ────▶│  prompt builder                    │◀── memory.search(query)
                │                                    │
                │  model thinks                      │
                │                                    │
                │  decision ──▶ tool execution       │
                │                                    │
   memory.write(fact)  ◀── orchestrator              │
                │                                    │
                └────────────────────────────────────┘
                                │
                                ▼
                       ┌──────────────────┐
                       │  Memory backend  │
                       │  (vector / SQL)  │
                       └──────────────────┘

The memory layer is behind the agent. The agent doesn't see backends or embeddings; it sees a search tool and (sometimes) a write tool. See Hermes Architecture Overview.

What Goes In

Don't memorize everything. Memory is signal storage, not a transcript dump.

Worth memorizing:

• User preferences and style choices
• Project facts (stack, conventions, deadlines)
• Decisions made in previous sessions
• Summaries of completed work
• Errors and their resolutions (so the agent doesn't repeat)

Not worth memorizing:

• Raw transcripts (use episodic memory or just logs)
• Tool results that were specific to one request
• Speculative thoughts the agent had mid-loop
• Anything sensitive (PII, secrets) — see Hermes Risks

A memory full of low-signal entries hurts retrieval. Garbage in, irrelevant retrieval out.

Memory Scope

A single agent's memory should be scoped:

• Per-agent — the researcher's memory is separate from the writer's
• Per-user — each user gets isolated memory (privacy + relevance)
• Per-project — work for project A doesn't leak into project B

Hermes models this as namespaces:

memory = VectorMemory(
    path="./mem",
    namespace=f"user_{user_id}/agent_{agent_name}",
)

Without scoping, you eventually have one giant pool of mixed memory and retrieval becomes useless. Scope from day one.

Read-Heavy vs Write-Heavy Agents

Match the backend to the pattern. See Hermes Memory Storage.

Memory ≠ RAG (But Related)

Retrieval-augmented generation (RAG) is a use case for memory: you have a knowledge base and you retrieve relevant chunks at query time. Memory in Hermes is a capability — RAG is one thing you can build with it. Other things:

• Personalization
• Cross-task learning
• Audit / compliance
• Conversation continuity
• Caching expensive computations

Treat RAG as a memory pattern, not the whole memory layer.

When You Don't Need Memory

Memory is overhead. Skip it when:

• Tasks are completely independent of one another
• The user doesn't return for follow-ups
• All needed context fits comfortably in the prompt
• Privacy or compliance forbids retention

A no-memory agent is simpler. Start without; add when a recurring "the agent should have remembered" pain point appears.

Common Mistakes

• Treating context as memory — they're different layers
• One global memory pool — leads to cross-contamination
• Memorizing everything — retrieval gets noisy
• No decay or pruning — memory grows unbounded
• Sensitive data unfiltered — PII landing in vector stores
• Writing the entire transcript — see Hermes Memory Optimization

Related

• Hermes Memory Storage — backends and write paths
• Hermes Memory Retrieval — read paths and ranking
• Hermes Memory Optimization — pruning, summarization, cost
• Hermes Context Management — how memory feeds context
• Hermes Knowledge Agents — RAG use case
• Hermes Index — full topic map