Things keeping me busy on weekends
Authors: Terry Chen, Kaiwen Che, Matthew Song
Despite advances in large language model (LLM) capability, their fundamental limitation of not being able to store context over long-lived interactions persists. In this paper, a novel human-inspired three-tiered memory architecture is presented that addresses these limitations through biomimetic design principles rooted in cognitive science. Our approach aligns the human working memory with the LLM context window, episodic memory with vector stores of experience-based knowledge, and semantic memory with structured knowledge triplets.
In comparison to traditional retrieval-augmented generation (RAG) techniques that store verbatim conversation segments, our system employs strategic memory consolidation procedures, abstracting key information into structured forms. Performance testing on the GoodAI Long-Term Memory benchmark demonstrates significant improvements in performance, with our memory-augmented GPT-4o achieving scores of up to 6.9/11 over the baseline 4.6/11. Additional testing across multi-agent domains demonstrates enhanced persistence and updating capacity of information.
State-of-the-art large language models (LLMs) possess remarkable natural language comprehension and generation. However, their architecture imposes tight constraints on memory retention and contextual comprehension during long-term interaction. Most existing LLMs operate within fixed context windows, typically ranging from 32,000 to 128,000 tokens, which impose inherent constraints on long-term conversation and complex reasoning tasks that span multiple turns.
The Baddeley and Hitch (1974, 2000) model of working memory provides a robust theoretical account of human information processing. The model presents memory as a multi-component system with central executive control of information flow, an episodic buffer of assembling memories into temporary experiences, a phonological loop of controlling verbal content, and a visuospatial sketchpad of controlling visual and spatial information.
Current approaches to increasing LLM memory capacity heavily rely on embedding-based retrieval-augmented generation (RAG). While the approach can deliver rapid access to previous data, it suffers greatly from issues like vector explosion, the unsustainable proliferation of embeddings as conversation history grows, lack of semantic structure in stored shreds, and difficulties in maintaining relations among relevant facts.
This work introduces a novel biomimetic approach to LLM memory extension that more accurately models the cognitive architecture of humans, with a three-tiered memory system distinguishing between immediate context, episodic memories, and semantic facts.
Our memory improvement system utilizes a three-layer architecture inspired by human cognition:
We divide the context window into two distinct segments:
Implemented as a vector database storing two forms of memory:
There are specialized components for:
We implemented the semantic memory schema as a structured class:
class SematicMemory(BaseModel):
"""Store all new facts, preferences, and relationships as triples."""
subject: str
predicate: str
object: str
context: str | None = None
Our episodic memory implementation stores experiential information with temporal context:
class EpisodicMemory(BaseModel):
"""Write the episode from the perspective of the agent within it."""
observation: str = Field(..., description="The context and setup - what happened")
thoughts: str = Field(
...,
description="Internal reasoning process and observations of the agent"
)
action: str = Field(
...,
description="What was done, how, and in what format."
)
result: str = Field(
...,
description="Outcome and retrospective."
)
The foundation of our strategy lies in sophisticated memory consolidation mechanisms that convert raw conversational information into structured memory representations:
Our semantic memory schema makes use of subject-predicate-object triples that eliminate episodic detail without sacrificing core relationships. Implementation follows several guiding principles:
Episodic memory stores full interactions in an ordered schema consisting of four main components:
GoodAI LTM benchmark results indicated radically better performance with our memory augmentation approach:
| Configuration | Score | Performance |
|---|---|---|
| Baseline GPT-4o | 4.6/11 | 41.8% |
| GPT-4o + Semantic Memory | 6.8/11 | 61.8% |
| GPT-4o + Episodic Memory | 6.9/11 | 62.7% |
| GPT-4o + Semantic & Episodic Memory | 6.0/11 | 54.5% |
These results reflect a general 20-percentage-point improvement in memory performance by our augmentation method. The differential performance aligns with the corresponding functional roles these types of memory serve in human cognition, wherein semantic memory enables fact recall and episodic memory enables experiential reasoning.
Our research provides empirical evidence for cognitive-inspired LLM memory enhancement methods. The witnessed performance improvements with three-tier memory architecture show that human memory systems offer valuable design concepts for overcoming inherent limitations in current AI designs.
The unexpected finding was the slightly worse performance of integrated memory systems compared to single implementations. This suggests complex interaction effects, which may mirror interference phenomena observed in human memory systems, where various forms of memory sometimes vie for mental resources.
Future research directions include:
This paper presents a novel biomimetic approach to enhancing LLM memory that addresses intrinsic limitations in current architectures. By embracing a three-level memory structure inspired by human cognitive processes, we demonstrate significant improvements in information retention, update, and context recall.
As LLMs advance towards more general intelligence capabilities, structured memory systems will play a larger role in enabling coherent long-term interactions, homogeneous knowledge states, and contextually appropriate information access. Our research contributes both pragmatic approaches for deploying this aspect of AI progress and theoretical frameworks to continue advancing this critical component of AI work.