Why context graphs?

How graph-structured memory compares to flat document stores and vector databases, and why it makes AI agents more capable.

AI agents need memory to be useful. But not all memory is created equal. This page explains why graph-structured memory — a context graph — is fundamentally better for agents than flat document stores or vector databases alone.

The problem with flat memory

Most agent frameworks store memory in one of three ways:

• Chat history — a linear sequence of messages
• Vector store — text chunks embedded and retrieved by similarity
• Key-value store — simple facts stored as string pairs

These approaches work for isolated question-answering. They break down when agents need to:

1. Connect information across conversations — “The patient I discussed yesterday has a new diagnosis.”
2. Reason about relationships — “Which organizations are connected to this person?”
3. Trace decisions back to evidence — “Why did the agent recommend this treatment?”
4. Navigate multi-hop paths — “Find all patients treated by doctors who trained at this hospital.”

A vector store can find similar things. It cannot find connected things. Those are different problems, and agents in production need both.

What a context graph is

A context graph is a Neo4j knowledge graph that combines three complementary memory layers in a single database:

Long-term memory

Persistent entities and relationships structured using the POLE+O model — the knowledge the system accumulates over time

Short-term memory

Conversation history per session, scoped by session_id, so the agent knows what was just said

Reasoning memory

Decision traces that record the full reasoning chain from question to answer, preserving auditable provenance

The agent doesn’t just retrieve documents — it navigates a structured knowledge space, following relationships and building context from connected entities.

The POLE+O model

Every context graph is built on the POLE+O entity classification system, defined in _base.yaml and shared across all domains:

Type	Label	Examples
Person	Person	Patients, employees, suspects, researchers
Organization	Organization	Companies, hospitals, teams, agencies
Location	Location	Facilities, regions, physical addresses
Event	Event	Encounters, incidents, transactions, appointments
Object	Object	Documents, medications, instruments, artifacts

Every domain ontology inherits these five base types and adds domain-specific subtypes on top. A healthcare domain adds Patient, Diagnosis, Treatment, and Medication. A financial services domain adds Account, Transaction, and Portfolio. The base structure stays constant; the domain vocabulary does not.

Base relationships are also inherited: WORKS_FOR (Person → Organization), LOCATED_AT (Organization → Location), and PARTICIPATED_IN (Person → Event) are available in every domain without redeclaring them.

Vector-only RAG vs. context graph

Capability	Vector-only RAG	Context graph
Semantic similarity search	Excellent	Good (with embeddings)
Exact relationship queries	Poor	Excellent
Multi-hop reasoning	Not possible	Native
Schema enforcement	None	Full
Explainable connections	No	Yes
Maintain conversation state	Requires external management	Built-in short-term memory
Audit past decisions	Not possible	Query historical traces by task, outcome, or date
Incremental updates	Requires re-embedding	Add nodes and edges

Context graphs do not replace vector search. Generated projects include a vector_client.py for semantic similarity queries. Graphs add structured knowledge and reasoning traces on top of vector retrieval.

Multi-hop reasoning

Graph traversal enables queries that are simply not possible with a flat store. Consider this Cypher query from the healthcare domain:

MATCH (p:Patient)-[:DIAGNOSED_WITH]->(d:Diagnosis)
MATCH (similar:Patient)-[:DIAGNOSED_WITH]->(d)
WHERE similar.patient_id <> $patient_id
OPTIONAL MATCH (similar)-[:HAD_ENCOUNTER]->(e:Encounter)-[:INCLUDES]->(t:Treatment)
RETURN similar, d, collect(DISTINCT t) AS treatments
LIMIT 10

This traverses four relationship hops to find patients with matching diagnoses and retrieve their treatments — a query that spans three entity types in a single database round-trip. There is no equivalent in a vector store.

Decision traceability

When an agent makes a high-stakes decision, the reasoning chain is preserved as a trace in the graph:

DecisionTrace → HAS_STEP → TraceStep[1] → TraceStep[2] → ...

Each step records the thought, the action taken (tool call and Cypher query), and the observation returned. This provides full reasoning provenance — critical for regulated domains like healthcare and finance where stakeholders must be able to answer: Why did the agent recommend this?

How create-context-graph implements this

When you scaffold a project, the generated application includes:

1. Neo4j schema

   Domain-specific constraints and indexes derived from your ontology YAML — entity types, uniqueness constraints, name indexes, and infrastructure indexes for Document and DecisionTrace nodes.

2. Agent tools

   Domain-specific Python functions that query the knowledge graph via Cypher. Each tool is generated from the agent_tools section of your ontology YAML and wired into whichever agent framework you chose.

3. Memory integration

   The context_graph_client.py initializes neo4j-agent-memory’s MemoryClient at startup, providing get_conversation_history() and store_message() for short-term memory across all 8 supported frameworks.

4. Decision trace storage

   The ingestion pipeline stores reasoning scenarios via client.reasoning.start_trace(), add_step(), and complete_trace(), building an audit trail that accumulates with every agent interaction.

5. Interactive visualization

   The Next.js frontend renders the live graph using NVL. It starts in schema view and updates incrementally as the agent executes tool calls — each tool_end SSE event triggers a graph refresh.

Next steps

Three memory types

Deep dive into short-term, long-term, and reasoning memory

Domain ontologies

How a single YAML file drives the entire generated application