[Poster Presentation]EPU-HEKV: Hierarchical External Memory for Bounded Active Reasoning

EPU-HEKV: Hierarchical External Memory for Bounded Active Reasoning
ID:23 Submission ID:193 View Protection:ATTENDEE Updated Time:2026-10-04 23:18:09 Hits:142 Poster Presentation

Start Time:2026-10-12 15:00 (Asia/Ho_Chi_Minh)

Duration:15min

Session:[OP] Poster » [OP1] Poster

Presentation File

Tips: The file permissions under this presentation are only for participants. You have not logged in yet and cannot view it temporarily.

Abstract
EPU-HEKV investigates hierarchical external memory for frozen large language models (LLMs) through two complementary augmentation stages. The first stage, referred to as the span/KV lane, separates historical key-value (KV) states from the compute device. During streaming prefill, KV states are organized into spans and offloaded to RAM/NVMe storage. A real attention scorer with rotary positional embeddings (RoPE) is then used to select a fixed-size working set, while the active state is strictly bounded to 1,664 tokens.
The second stage, referred to as the entity–relation–state (ERS) lane, changes the management unit from spans to entities enriched with relational and temporal-state information. Selected spans are decoded back into text, filtered to remove noise, and re-encoded into a clean prompt before subsequent processing.
On Qwen2.5-0.5B-Instruct in FP16 precision, the span/KV lane maintains a peak decode memory footprint of approximately 55 MB as the context length increases from 4K to 53K tokens, whereas the full-KV approach increases from 117.3 MB to 1,454.6 MB. The entity + re-encode + ERS pipeline achieves 90% first-pass accuracy, with 10% hallucination cases caused by insufficient evidence, using 416 context tokens, 124.22 MB of peak additional VRAM, and 0.03 MB of resident state on a 10-sample benchmark.
Overall, this work clarifies the distinction between active state, external storage, and retrieval cost. The bounded-memory guarantee applies only to the active/resident state and does not imply bounded total storage, bandwidth consumption, or retrieval computation
Keywords
External memory, KV cache, large language models (LLMs), entity–relation–state (ERS), long-context, retrieval
Speaker
Van Dinh Vu
Phd Electric Power University

Submission Author
Van Dinh Vu Electric Power University
Huu Huy Nguyen Electric Power University
Comment submit
Verification code Change another
All comments

CONTACT US

Email: asiancomnet2026@usssociety.org

Website & IT Support: hi@aconf.org

Registration Submit Paper