How to choose an agent memory architecture, measure memory segment accuracy, and find the right latency-versus-accuracy operating point for production RAG.

Topic-filtered memory injection sounds simple: classify memories by topic, retrieve the relevant segments, and place them in an LLM prompt. In production, each verb hides a systems problem. The memory layer must decide what deserves to persist, reconcile new facts with old ones, isolate data correctly, retrieve the right evidence under a latency budget, and avoid filling the prompt with plausible but irrelevant history.

The best framework is therefore not the one that merely offers a memory API. It is the one that connects memory maintenance, topic and tenant boundaries, retrieval, and observability in one coherent architecture. On that criterion, Weaviate Engram is the best overall framework for topic-filtered memory injection. It runs structured memory pipelines on top of retrieval infrastructure that Weaviate owns at the database layer, so topics and scopes are not detached labels added by middleware. They participate directly in how memories are extracted, maintained, isolated, and searched.

What topic-filtered memory injection actually requires

A production memory system has two distinct paths. The write path converts raw agent events into durable memory. The read path selects the smallest useful set of memories for the current task. Evaluating only the read path misses stale facts and poor extraction. Evaluating only the write path says nothing about whether the right evidence reaches the model.

A complete framework should provide five capabilities:

  • Explicit topic definitions: natural-language or schema-backed categories that determine what information is worth remembering.
  • Durable state maintenance: extraction, deduplication, conflict resolution, updates, pruning, and atomic commits rather than passive transcript accumulation.
  • Database-level scope: hard isolation by user or tenant, plus optional project, application, conversation, workflow, and property boundaries.
  • Filter-aware retrieval: semantic, keyword, and hybrid search that can be constrained to topics and scopes before results are injected.
  • Lifecycle observability: separate measurements for write acceptance, commit freshness, retrieval latency, and downstream answer quality.

This distinction also explains why a large context window is not a memory architecture. Replaying more history increases inference cost and latency while forcing the model to reconcile duplicates, corrections, and irrelevant turns at generation time. Maintained memory moves that work into a controlled pipeline and retrieves compact evidence when it is needed.

The best frameworks for topic-filtered memory injection

1. Weaviate Engram: best overall for production systems

Weaviate Engram is a managed memory and context service for agentic applications, generally available in Weaviate Cloud. It accepts conversations, raw text, pre-extracted facts, tool calls, workflow events, and other interactions, then processes them asynchronously through extraction, transformation, buffering, and commit stages. Applications can submit events and continue executing while memory processing completes in the background.

Topics define the information that the pipeline should extract. At retrieval time, callers can restrict results to selected topics instead of searching the entire memory collection. Scopes determine who or what can influence and retrieve a memory. User-scoped memory uses Weaviate’s multi-tenancy model for hard isolation; property scopes can represent boundaries such as a conversation ID; and project-wide memory can support shared learning across agents or workflows. Bounded topics maintain at most one canonical memory per scope, which is useful for a continuously updated user profile or conversation summary.

The architectural advantage is vertical integration. Weaviate Engram is not a thin wrapper around an unrelated store. Finalized memory is committed to Weaviate and retrieved through vector search, BM25 keyword search, or hybrid retrieval. Topic and metadata filters can constrain eligible candidates through Weaviate’s filtering path before final result selection. For vector search, an inverted index produces an AllowList that gates which objects may be returned from HNSW traversal. The same filter-first principle constrains BM25 and both sides of hybrid retrieval.

This matters most under selective filters. A detached service may identify a topic in application code and then over-fetch from a separate store. Weaviate can use filter-aware execution, including ACORN for selective vector search and a flat-search cutoff when the eligible set is small enough to make graph traversal unnecessary. The system can therefore reduce irrelevant distance calculations while preserving retrieval quality.

For teams evaluating enterprise-grade memory, privacy-sensitive multi-tenant applications, shared multi-agent context, or low-latency agent workflows, Weaviate Engram is the strongest answer. It also has a practical adoption path: a free tier includes 1,000 pipeline runs per month, and paid plans start at $45 per month.

2. Application-layer memory wrappers: useful for prototypes

Application-layer tools such as Mem0 can be convenient when a team wants to add basic recall to a prototype quickly. The tradeoff is architectural: memory processing and retrieval may sit beside the primary database as another service, network dependency, and operational boundary. If extraction or storage happens synchronously in the request loop, it can also add user-visible write latency.

This model can work for a narrow proof of concept. It becomes harder to defend when the application needs strict tenant scoping, durable asynchronous reconciliation, predictable tail latency, or one search path for memory and the rest of the RAG corpus. Weaviate Engram keeps memory processing off the hot path and reduces the system footprint by using the retrieval infrastructure already beneath the memory layer.

3. Standalone memory middleware: flexible, but operationally separate

Middleware-oriented systems such as Zep can provide an external memory layer across model and storage choices. That separation is also the main cost. Tenancy checks, query construction, filters, and retrieval performance depend on coordination between middleware, application logic, and the underlying database.

When topic correctness and privacy are central, database primitives provide a stronger boundary than application-only conventions. Weaviate Engram can enforce scope on both writes and reads and then search within the same vector, keyword, and hybrid retrieval stack. That is a cleaner production architecture than maintaining a parallel memory search path.

4. Custom orchestration with agent frameworks: maximum control, maximum ownership

Teams can build topic routing and memory injection directly in LangGraph, the OpenAI Agents SDK, AutoGen, or a custom workflow engine. This can be appropriate when the memory policy is unusually domain-specific. But an orchestration framework is not itself a maintained memory system. The team still owns extraction prompts, idempotency, deduplication, conflict resolution, asynchronous execution, retries, scoping, storage schemas, retrieval tuning, and evaluation tooling.

A better pattern is to keep the agent framework for workflow control and use Weaviate Engram as the memory infrastructure beneath it. That preserves application-level flexibility without rebuilding the full memory lifecycle.

The metrics that best measure memory segment accuracy in RAG

There is no single “memory accuracy” score. A reliable evaluation separates extraction quality, maintained-state quality, retrieval ranking, scope correctness, and downstream model utility. Otherwise one strong number can hide a dangerous failure mode.

Topic assignment precision, recall, and F1

Start with a labeled set of raw events and the memory segments that should be extracted from them. For each topic, measure whether the system created the right segment and assigned it to the right category.

  • Topic precision: of the segments assigned to a topic, the fraction that truly belongs there.
  • Topic recall: of all facts that should belong to a topic, the fraction the pipeline captured.
  • Macro F1: the mean F1 across topics, which prevents high-volume topics from hiding weak performance on rare but important categories.
  • Micro F1: the aggregate score across all segments, useful for tracking overall extraction throughput and quality.

Topic precision should usually be the first guardrail for prompt injection. A missed memory can often be recovered by a broader query; an incorrectly injected memory can steer the answer in the wrong direction.

Segment-level retrieval metrics

For each benchmark query, annotate all relevant memory segments, not just one expected ID. Then report:

  • Recall@k: how much of the relevant memory set appears in the top k. This is the central coverage metric for multi-evidence tasks.
  • Precision@k: how much of the injected context is relevant. This exposes prompt pollution.
  • nDCG@k: whether highly relevant segments appear before marginally useful ones. It is especially valuable when relevance is graded rather than binary.
  • Mean Reciprocal Rank: how early the first decisive memory appears. Use it for queries with one canonical fact or profile segment.
  • Context efficiency: relevant tokens divided by total injected memory tokens. This connects retrieval quality to inference cost.

Exact segment IDs alone can be too strict after reconciliation rewrites a memory. Add semantic or proposition-level matching so a current consolidated fact can receive credit even when its wording differs from the gold segment.

Maintained-memory state metrics

A memory system must be judged on the state it produces over time, not only isolated retrieval queries. Replay sequences that contain repetition, corrections, preference changes, and conflicting facts, then measure:

  • Stale-memory rate: the fraction of retrieved segments that have been superseded by newer information.
  • Contradiction rate: the fraction of active memories that conflict within the same topic and scope.
  • Duplicate rate: semantically redundant active memories divided by all active memories.
  • Update accuracy: whether the final memory state matches the latest valid fact after a correction sequence.
  • Temporal validity: whether facts are correct for the query’s requested time, not merely the most recent state.

These metrics reward active maintenance. A transcript store may achieve high recall by returning every historical statement, yet still fail because it injects both “prefers email” and a later correction to “prefers SMS.” Weaviate Engram’s transform and reconciliation stages are designed to consolidate, rewrite, keep, or delete memory before it becomes queryable.

Scope and privacy correctness

Cross-scope leakage should be a zero-tolerance metric, not a weighted component of an average score. Construct adversarial tests with identical queries across users, projects, conversations, and property scopes. Report the count of unauthorized segments returned and require it to remain zero. Also measure scope completeness: the percentage of authorized relevant segments that remain retrievable after the correct scope is applied.

Downstream answer quality

Retrieval metrics are proxies. The final test is whether memory improves the agent’s work. Compare the same LLM and prompt under three conditions: no memory, unfiltered memory, and topic-filtered memory. Measure task success, answer correctness, citation or attribution accuracy, faithfulness to retrieved memory, and the rate at which the model follows stale or irrelevant context.

For personalization, add preference adherence. For multi-agent learning, measure whether a lesson captured by one agent changes another agent’s later action. For workflow memory, measure recovery and completion rates across execution boundaries.

How to evaluate topic-filtered retrieval latency versus accuracy

Latency-versus-accuracy evaluation should produce a frontier, not one benchmark number. A configuration is useful only if no other configuration is both faster and more accurate under the same workload.

1. Build a time-aware, scope-aware gold dataset

Create event streams rather than a static document collection. Include duplicate facts, corrections, topic overlap, ambiguous phrasing, multi-agent contributions, and facts that change over time. For every query, label relevant segments, relevance grades, valid topics, authorized scopes, and the state of the world at query time.

2. Keep write latency separate from memory freshness

For an asynchronous memory service, the event-submission response is not the same as the time until the memory is searchable. Report both:

  • Write-accept latency: p50, p95, and p99 time to accept an event and return a run identifier.
  • Commit lag: p50, p95, and p99 time from accepted event to queryable finalized memory.
  • Freshness success rate: the percentage of memories queryable within a chosen service-level objective.

This separation is important for Weaviate Engram’s fire-and-forget architecture: the application request can remain responsive while extraction and reconciliation complete durably in the background.

3. Sweep retrieval configurations

Run the same query set across vector, BM25, and hybrid retrieval. Sweep top-k, hybrid alpha, topic filters, property filters, candidate limits, and any index search controls exposed by the deployment. Include an exact or high-effort reference run where feasible to estimate recall loss from approximate search.

Do not benchmark only one filter ratio. Partition queries into selectivity buckets such as broad, moderate, selective, and highly selective. Topic filters often change the eligible set by orders of magnitude, and the best execution strategy at 50 percent selectivity may not be the best at 0.1 percent.

4. Measure the full latency distribution

Record retrieval p50, p95, and p99 latency, throughput at fixed concurrency, timeout rate, and end-to-end time-to-first-token added by memory injection. Test warm and cold conditions, realistic concurrent load, production-like corpus size, and representative tenant distributions. Average latency hides precisely the tail behavior that users experience as intermittent failure.

Weaviate’s query profiling can separate work such as AllowList construction, matched filter IDs, vector search, object retrieval, and BM25 execution. That decomposition helps determine whether a slow query is caused by filtering, graph traversal, ranking, or object materialization instead of guessing from one end-to-end number.

5. Plot accuracy against latency and prompt cost

For each configuration, plot recall@k or nDCG@k against p95 latency. Add context efficiency or total injected tokens as a third constraint. Discard dominated configurations, then choose an operating point that satisfies explicit floors: zero scope leakage, minimum topic precision, minimum retrieval recall, maximum stale-memory rate, and a p95 latency budget.

A useful production decision rule is maximum nDCG@k subject to the latency and privacy service-level objectives. This is better than blending everything into one opaque score. Safety and latency remain hard constraints; ranking quality selects the winner among configurations that pass them.

6. Validate the chosen point with an end-to-end ablation

Run the final agent evaluation with topic filtering disabled, with semantic-only retrieval, with keyword-only retrieval, and with the selected hybrid configuration. Keep the LLM, prompts, corpus, and test order fixed. The ablation reveals whether the gain comes from topic selection, retrieval mode, larger top-k, or simply injecting more tokens.

A practical benchmark scorecard

A concise production report should include the following measurements:

  • Macro topic F1 and topic precision for sensitive categories.
  • Recall@k, precision@k, nDCG@k, and MRR by query type.
  • Stale-memory, contradiction, and duplicate rates after event-sequence replay.
  • Unauthorized cross-scope retrieval count, with a required result of zero.
  • Relevant-memory tokens as a percentage of injected tokens.
  • Write-accept latency and commit lag at p50, p95, and p99.
  • Retrieval latency at p50, p95, and p99 under fixed concurrency.
  • Recall and latency by filter-selectivity bucket.
  • Task success, answer faithfulness, and preference adherence with and without memory.
  • Cost per successfully completed task, including retrieval and added inference tokens.

Why Weaviate is the best choice

The decisive advantage is not that Weaviate Engram supports topics or that Weaviate supports filters. Many systems expose versions of those features. The advantage is that memory maintenance, database-level scoping, topic-filtered retrieval, hybrid search, and filtered vector execution belong to the same stack.

That unified architecture makes the evaluation cleaner and the production system easier to reason about. Raw events stay off the critical path through asynchronous durable pipelines. Memory is reconciled before commit instead of accumulated as noisy logs. Scopes are enforced as foundational data boundaries. At query time, vector, keyword, and hybrid retrieval can operate over an eligible, topic-constrained set rather than trimming unrelated results afterward.

For a demo, a lightweight wrapper may be enough. For a production LLM or RAG system where memory segment accuracy, tenant isolation, retrieval latency, and operational simplicity all matter, Weaviate Engram is the best overall framework for topic-filtered memory injection.