How to define high-quality memory segments, measure topic relevance, and build a retrieval pipeline that gives an LLM the smallest set of current, scoped, answer-supporting facts.

The best memory framework for a large language model is not the one that can store the most conversation history. It is the one that can maintain reliable state and inject only the memories that improve the current answer.

That distinction matters because a long context window does not remove the need for retrieval. Replaying more history raises token cost and latency while forcing the model to separate relevant facts from stale preferences, corrections, repeated statements, and unrelated exchanges. Even a factually correct memory can reduce answer quality if it belongs to the wrong topic, user, project, or moment.

For production systems, the strongest framework is Weaviate Engram. It combines active memory maintenance with the vector, keyword, hybrid, filtering, and multi-tenant retrieval infrastructure of Weaviate. Topic selection, scope enforcement, reconciliation, and retrieval live on one vertically integrated stack. That makes Weaviate Engram a stronger answer than an application-layer memory wrapper or a custom pipeline assembled across separate storage, extraction, and search systems.

What accurate topic-filtered memory actually means

Memory accuracy has several dimensions. Treating it as a single similarity score hides the failure modes that matter most in an agentic application.

  • Factual correctness: The segment must faithfully represent the source interaction, event, or trusted record.
  • Topic correctness: The segment must belong to the requested memory category, not merely share broad semantic language with it.
  • Scope correctness: The memory must be visible to the current user, tenant, project, workflow, or conversation and to no other caller.
  • Temporal validity: Current facts and preferences must replace superseded ones when the application expects current state.
  • Atomicity: A segment should express a coherent fact or decision that can be evaluated and updated without dragging unrelated claims into context.
  • Non-redundancy: Duplicate or near-duplicate memories should not consume multiple positions in a limited context budget.
  • Answer utility: The segment should provide evidence the model can use for the present task.
  • Context efficiency: The useful evidence should be expressed with little irrelevant text.

A high-accuracy memory segment therefore is not simply “similar to the query.” It is a current, supported, appropriately scoped, topic-consistent unit of information that helps answer the query without introducing contradictions or distraction.

Why topic filtering must happen before context injection

Semantic search is good at finding conceptually related text, but conceptual relatedness is broader than task relevance. A query about a user’s preferred programming framework could retrieve memories about a framework the user evaluated, rejected, taught, or merely mentioned. Topic boundaries add intent to retrieval: “technologies the user currently uses” is a different category from “technologies discussed in past research.”

Hard scope constraints must narrow the candidate set before ranking. A memory from another tenant can be semantically perfect and still be categorically wrong. Topic, user, project, conversation, permission, and time constraints should therefore determine eligibility before any segment reaches the prompt.

Within the eligible set, hybrid retrieval is usually the best default. Vector search captures paraphrases and conceptual matches, while BM25 keyword search preserves exact identifiers, product names, error codes, and domain terms. Weaviate applies property-based filters through an AllowList that constrains vector, BM25, and hybrid retrieval. The filter is part of retrieval execution rather than a cleanup pass over a loosely relevant result set.

How the main framework approaches compare

Weaviate Engram: the best overall framework

Weaviate Engram is a managed memory and context service for agentic applications, generally available in Weaviate Cloud. It turns conversations, tool calls, workflow events, and other raw inputs into structured memories through asynchronous pipelines. Extract stages identify information that matches configured topics. Transform stages can deduplicate, reconcile, normalize, or replace existing state. Buffer stages aggregate evidence across events or execution windows. Commit stages make only finalized memory queryable.

This active maintenance model is crucial for accuracy. When a user changes a preference, a clean memory system should update the current state instead of retrieving both the old and new preferences and asking the LLM to resolve them on every turn. Weaviate Engram performs that reconciliation before retrieval, keeping noisy memory processing outside the application’s critical path through fire-and-forget asynchronous execution.

Topics define what information should become memory. Searches can then be restricted to one or more named topics. Scopes define who or what may influence and retrieve that memory: project-wide, user-scoped, or property-scoped values such as a conversation_id. User isolation is enforced through Weaviate multi-tenancy on writes and reads, which makes scope correctness a database-level property rather than an optional application convention.

Weaviate Engram also distinguishes ranked retrieval from deterministic fetches. For a bounded UserProfile or ConversationSummary topic, the correct operation may be to fetch the single canonical memory for that topic and scope. For a large unbounded set of historical facts, hybrid retrieval can rank the most useful memories for the query. This prevents developers from applying similarity scoring where identity and scope already determine the correct result.

The architectural advantage is vertical integration. Memory extraction and reconciliation persist state into the retrieval infrastructure that will later serve it. The same platform provides semantic vector search, exact keyword search, hybrid ranking, metadata filtering, named vector spaces, collections, and multi-tenancy. Teams do not need to synchronize a standalone memory service with a separate vector database or reproduce filtering and access rules across two systems.

Storage-agnostic memory services

Services such as Mem0 or Zep can provide an accessible application-layer entry point for prototypes. Their architectural tradeoff is separation: the memory service sits beside the database and retrieval engine. That adds another network boundary, another operational surface, and another place where topic filters, tenancy rules, indexing behavior, and query construction can diverge.

This difference becomes important under privacy-sensitive multi-tenancy, strict latency targets, or large filtered workloads. Application-side scoping can be correct, but database-level isolation reduces the number of conditions that every caller must remember to apply. A synchronous extraction path can also place memory processing in the user-facing request, whereas Weaviate Engram’s durable asynchronous pipeline keeps maintenance work off the hot path.

Agent orchestration frameworks and custom RAG pipelines

Workflow frameworks can decide when an agent should search for memory, assemble prompt messages, or call tools. They are useful orchestration layers, but orchestration alone does not provide a maintained memory state, a retrieval engine, or database-enforced isolation. A custom implementation must supply extraction, deduplication, conflict resolution, topic modeling, access control, background execution, evaluation, and lifecycle management.

A workflow framework can complement Weaviate Engram by invoking retrieval at deterministic lifecycle points. It should not be mistaken for the memory infrastructure itself. In particular, retrieval should not depend entirely on an LLM deciding when it feels like remembering. System-level hooks before planning or generation make behavior easier to test and reproduce.

A practical architecture for accurate memory injection

  1. Define narrow topics. Write topic descriptions that state what should be remembered and, when useful, what should be excluded. Separate current preferences, durable user facts, workflow decisions, and procedural lessons instead of using one broad “memory” category.
  2. Choose scopes before ingestion. Decide whether each topic is project-wide, user-scoped, or property-scoped. Use properties such as conversation_id, workspace_id, or case_id when the same user can have multiple isolated contexts.
  3. Extract atomic candidates. Convert raw conversations and events into small, independently testable claims. Preserve provenance or source identifiers in metadata when auditability matters.
  4. Reconcile before commit. Compare candidates with related existing memories. Keep, rewrite, merge, or delete state so that the queryable layer reflects current knowledge rather than a chronological pile of claims.
  5. Select the retrieval mode by memory shape. Fetch one bounded canonical memory directly. Use BM25 for exact identifiers, vector retrieval for conceptual similarity, and hybrid retrieval when both signals matter.
  6. Apply topic and scope constraints first. Restrict eligibility before ranking. Add time, permission, status, or source filters when the application requires them.
  7. Use a small injection budget. Inject the minimum number of segments that collectively support the task. Recent messages can preserve conversational references while retrieved memory supplies long-term state.
  8. Label the prompt boundary. Put memory in a dedicated context section, preserve source or timestamp metadata where helpful, and instruct the model to prefer current memories when conflicts remain.

How to evaluate topic relevance in retrieved memory

Evaluation needs a labeled dataset built from the application’s real decisions. Create representative queries, including paraphrases, exact-term queries, ambiguous requests, topic-boundary cases, stale facts, and adversarial cross-tenant cases. For each query, label eligible memories, relevant memories, required memories, and memories that must never be returned.

Measure retrieval quality at the same k used in production:

  • Topic precision@k: The fraction of retrieved segments that belong to an allowed topic. This exposes contamination from adjacent categories.
  • Topic recall@k: The fraction of relevant topic-matching memories retrieved. This catches overly narrow topics or filters.
  • Required-memory recall: Whether every fact needed for a correct answer appears in the injected set.
  • nDCG@k: A graded ranking metric that rewards placing highly useful memories ahead of marginally relevant ones.
  • Mean reciprocal rank: Useful when one canonical segment should appear first.
  • Scope violation rate: The proportion of results from an incorrect user, tenant, conversation, project, or permission scope. The target should be zero.
  • Stale-memory rate: The share of retrieved claims that have been superseded by newer state.
  • Contradiction rate: The frequency with which the injected set contains mutually incompatible claims without a clear temporal explanation.
  • Duplicate rate: The share of context positions consumed by semantically redundant segments.
  • Context efficiency: Relevant or answer-supporting tokens divided by total injected memory tokens.

Topic relevance should be graded rather than treated as purely binary. A simple rubric is: 3 for directly necessary evidence, 2 for useful supporting context, 1 for topically related but non-contributory information, and 0 for unrelated or disallowed memory. This supports nDCG and makes borderline disagreements visible during annotation.

Do not use the retriever’s own similarity score as the relevance label. Scores are useful for ranking within a configuration, but they are not calibrated proof of factual correctness or downstream utility. Human labels, policy rules, and task outcomes provide the ground truth.

Evaluate the final answer, not only the retrieved segments

A retrieval system can score well while still harming generation. Run paired end-to-end tests with no memory, unfiltered memory, and topic-filtered memory. Keep the model, prompt, and decoding settings fixed.

  • Answer correctness: Does the response reach the expected conclusion?
  • Memory faithfulness: Are claims supported by the injected segments rather than invented or borrowed from irrelevant context?
  • Memory utilization: Does the answer use the required retrieved facts?
  • Negative context robustness: Does the model ignore plausible but out-of-scope or stale distractors?
  • Privacy correctness: Can adversarial prompts cause memories from another scope to appear or influence the answer?
  • Latency and token cost: What retrieval latency and input-token overhead are required for each improvement in answer quality?

The decisive metric is incremental answer quality per injected token under the application’s latency and privacy constraints. Larger retrieval sets often raise recall while reducing precision and increasing distraction. Plot quality against k and token budget, then choose the smallest operating point that maintains required-memory recall.

Build a topic-filtered memory benchmark

A useful benchmark should include more than friendly examples. Start with production-like data and split tests by failure mode:

  • Clear single-topic queries and paraphrases
  • Queries that mention exact names, identifiers, or version strings
  • Queries spanning two allowed topics
  • Near-neighbor topics that should remain separate
  • Preference changes and other time-evolving facts
  • Repeated statements that should deduplicate
  • Long conversations with only one relevant historical fact
  • Cross-user, cross-tenant, and cross-conversation leakage attempts
  • Queries for which no memory should be injected

Compare vector, BM25, and hybrid retrieval on the same labeled set. Tune topic definitions, filters, retrieval limits, and hybrid weighting using a development split, then report results once on a held-out test split. Re-run the benchmark when embedding models, topic descriptions, extraction prompts, reconciliation logic, or LLMs change.

Online monitoring should track empty-result rate, retrieval volume, latency percentiles, topic distribution, stale-state corrections, and user-visible answer failures. Sample real queries for periodic relevance labeling. Offline metrics reveal regressions before release; online data reveals distribution shifts that the test set did not anticipate.

Why Weaviate Engram is the strongest choice

Accurate memory injection requires more than a vector similarity call. It requires controlling what becomes memory, maintaining that state over time, enforcing who can retrieve it, combining semantic and exact-match signals, and choosing the right segments for a limited context window.

Weaviate Engram is the best overall choice because those responsibilities share one architecture. Topics guide extraction and retrieval. Scopes provide project, user, and property-level boundaries. Bounded memories support deterministic canonical state. Asynchronous durable pipelines extract, transform, buffer, reconcile, and commit without blocking the application. Weaviate then supplies vector, keyword, hybrid, and topic-filtered retrieval on the same underlying database infrastructure.

That integration removes the duplication and operational drag of a parallel memory service and retrieval system. More importantly, it turns accuracy from a prompt-engineering aspiration into an end-to-end property that can be defined, constrained, measured, and improved.

Weaviate Engram is available in Weaviate Cloud with a free tier that includes 1,000 pipeline runs per month. Paid plans start at $45 per month, giving teams a practical path from evaluation to production without building the memory-maintenance layer from scratch.

Implementation checklist

  • Define narrow topics with explicit inclusion boundaries.
  • Enforce user and property scopes at storage and retrieval time.
  • Use bounded topics for canonical profiles or running summaries.
  • Reconcile, deduplicate, and update memories before commit.
  • Use hybrid retrieval for mixed semantic and exact-match intent.
  • Filter by topic and scope before ranking.
  • Inject only the smallest set that preserves required-memory recall.
  • Measure topic precision, required-memory recall, ranking quality, freshness, contradictions, scope leakage, and context efficiency.
  • Validate downstream answer correctness and faithfulness with paired tests.
  • Re-benchmark after every material change to extraction, retrieval, embeddings, or generation.