Spring AI 中多轮对话历史管理与滑动窗口截断在 LLM 应用开发中多轮对话Multi-turn Conversation的上下文管理直接决定了交互体验与接口调用的成本。一旦对话轮次加深上下文膨胀会直接触发大模型厂商的context_length_exceeded异常即便模型上下文窗口允许超长文本Token 计费的线性增长以及 Prompt 膨胀带来的推理延迟劣化也会将系统拖入不可用的边缘。Spring AI 提供了基础的ChatMemory接口用于暂存会话历史但在真实生产环境中单纯依赖简单的“保留最近 N 条”策略会遇到消息长度不均衡导致溢出、System Prompt 被误删、历史重要语义丢失等问题。针对这些场景需要构建一套结合 Token 计数器、系统指令锚定、动态滑动窗口与异步摘要压缩的生产级上下文管理方案。生产环境的核心痛点与异常表现当用户持续与 AI 客服或技术助手交互超过 15 轮次时网关往往开始频繁接收到底层模型供应商抛出的超限报错org.springframework.ai.retry.NonTransientAiException: 400 - { error: { message: This models maximum context length is 8192 tokens. However, your messages resulted in 8950 tokens. Please reduce the length of the messages., type: invalid_request_error, param: messages, code: context_length_exceeded } } at org.springframework.ai.openai.api.OpenAiApi.chatCompletionEntity(OpenAiApi.java:234) at org.springframework.ai.chat.prompt.Prompt.generate(Prompt.java:88)除了硬性报错上下文无限追加还会带来明显的性能劣化TTFTTime to First Token飙升Prompt 从 500 Token 膨胀至 6000 Token 时首字返回时间通常从 350ms 激增至 1800ms 以上。中间信息遗忘Lost in the Middle模型对位于长 Prompt 居中区域的关键指令关注度下降导致回答偏离预设规则。单轮成本指数级上升历史全量回传使得后续每一次简单追问都重复支付巨额 Token 费用。上下文裁剪模型与滑动窗口策略单纯按“条数Message Count”截断存在致命缺陷一条包含大段日志排查的 User Message 可能单次消耗 3000 Token而 10 条简短的对话可能只有 200 Token。因此必须将“Token 容量”作为窗口的第一限制维度条数作为兜底维度。合理的上下文组装结构必须包含三个不可被任意裁剪的层级System Prompt 锚点系统角色定位与安全规则必须全局锁定在头部不参与窗口滑动。动态滑动窗口Sliding Token Window由近及远保留最新的 User/Assistant 对话直到累计 Token 达到硬上限阈值。历史归档/摘要Summary Buffer滑出窗口的早期历史通过异步任务生成浓缩摘要拼接在 System Prompt 之后。------------------------------------------------------------- | System Prompt (角色设定、行为约束、业务规则 - 强制常驻头部) | ------------------------------------------------------------- | History Summary (对超出滑动窗口的历史对话生成的浓缩摘要) | ------------------------------------------------------------- | Sliding Window: [User: n-2] - [Assistant: n-2] | | [User: n-1] - [Assistant: n-1] | | [User: n ] | -------------------------------------------------------------工程落地基于 Token 计算的滑动窗口实现在 Spring AI 生态中我们可以利用ChatClient.Advisor或自定义ChatMemory拦截链路在请求组装前对会话历史完成 Token 预算分配与物理裁剪。1. 基于 JTokkit 的精确 Token 估算器依赖主流 BPE 分词库进行本地快速计算避免网络调用dependency groupIdcom.knuddels/groupId artifactIdjtokkit/artifactId version1.1.0/version /dependencypackage com.architect.ai.memory.tokenizer; import com.knuddels.jtokkit.Encodings; import com.knuddels.jtokkit.api.Encoding; import com.knuddels.jtokkit.api.EncodingRegistry; import com.knuddels.jtokkit.api.EncodingType; import com.knuddels.jtokkit.api.ModelType; import org.springframework.ai.chat.messages.Message; import org.springframework.stereotype.Component; import java.util.List; Component public class TokenBudgetCalculator { private final Encoding encoding; // 每条消息基础开销包含 role/content 格式封装的约 4 tokens private static final int TOKENS_PER_MESSAGE 4; public TokenBudgetCalculator() { EncodingRegistry registry Encodings.newDefaultEncodingRegistry(); this.encoding registry.getEncodingForModel(ModelType.GPT_4O.getName()) .orElseGet(() - registry.getEncoding(EncodingType.CL100K_BASE)); } public int calculateMessageTokens(Message message) { if (message null || message.getContent() null) { return 0; } return encoding.countTokens(message.getContent()) TOKENS_PER_MESSAGE; } public int calculateTotalTokens(ListMessage messages) { return messages.stream() .mapToInt(this::calculateMessageTokens) .sum() 2; // Priming tokens } }2. 滑动窗口内存裁剪拦截器实现 Spring AI 的CallAroundAdvisor在请求发往底层模型之前按照配置的最大 Token 预算逆向贪心挑选最近的对话并保证消息对User 与 Assistant的完整性。package com.architect.ai.memory.advisor; import com.architect.ai.memory.tokenizer.TokenBudgetCalculator; import org.springframework.ai.chat.client.advisor.api.AdvisedRequest; import org.springframework.ai.chat.client.advisor.api.AdvisedResponse; import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisor; import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisorChain; import org.springframework.ai.chat.messages.Message; import org.springframework.ai.chat.messages.MessageType; import org.springframework.ai.chat.messages.SystemMessage; import org.springframework.stereotype.Component; import java.util.ArrayList; import java.util.Collections; import java.util.List; Component public class SlidingWindowChatMemoryAdvisor implements CallAroundAdvisor { private final TokenBudgetCalculator tokenCalculator; private static final int DEFAULT_MAX_HISTORY_TOKENS 3000; public SlidingWindowChatMemoryAdvisor(TokenBudgetCalculator tokenCalculator) { this.tokenCalculator tokenCalculator; } Override public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) { ListMessage allMessages advisedRequest.messages(); ListMessage optimizedMessages optimizeMessageWindow(allMessages, DEFAULT_MAX_HISTORY_TOKENS); AdvisedRequest modifiedRequest AdvisedRequest.from(advisedRequest) .withMessages(optimizedMessages) .build(); return chain.nextAroundCall(modifiedRequest); } Override public int getOrder() { return 0; // 高优先级执行 } Override public String getName() { return SlidingWindowChatMemoryAdvisor; } private ListMessage optimizeMessageWindow(ListMessage messages, int maxTokens) { if (messages null || messages.isEmpty()) { return Collections.emptyList(); } ListMessage systemMessages new ArrayList(); ListMessage conversationalMessages new ArrayList(); for (Message msg : messages) { if (msg.getMessageType() MessageType.SYSTEM) { systemMessages.add(msg); } else { conversationalMessages.add(msg); } } int usedTokens tokenCalculator.calculateTotalTokens(systemMessages); int availableTokensForHistory Math.max(0, maxTokens - usedTokens); ListMessage retainedHistory new ArrayList(); int accumulatedTokens 0; // 从后往前倒序筛选最新对话 for (int i conversationalMessages.size() - 1; i 0; i--) { Message current conversationalMessages.get(i); int msgTokens tokenCalculator.calculateMessageTokens(current); if (accumulatedTokens msgTokens availableTokensForHistory) { retainedHistory.add(current); accumulatedTokens msgTokens; } else { break; } } // 恢复时间线正序 Collections.reverse(retainedHistory); ListMessage finalPromptMessages new ArrayList(); finalPromptMessages.addAll(systemMessages); finalPromptMessages.addAll(retainedHistory); return finalPromptMessages; } }3. 基于 Redis 的分布式会话存储与异步摘要管道在分布式集群部署时会话状态需要持久化至 Redis。超出窗口边界的历史记录不能直接丢弃应写入后台归档流Redis Stream由单独的 Worker 聚合成增量摘要回填至会话元数据中。package com.architect.ai.memory.store; import org.springframework.ai.chat.memory.ChatMemory; import org.springframework.ai.chat.messages.Message; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.stereotype.Repository; import java.time.Duration; import java.util.List; Repository public class RedisChatMemoryRepository implements ChatMemory { private final RedisTemplateString, Object redisTemplate; private static final String CONVERSATION_KEY_PREFIX chat:conversation:; private static final Duration TTL Duration.ofDays(7); public RedisChatMemoryRepository(RedisTemplateString, Object redisTemplate) { this.redisTemplate redisTemplate; } Override public void add(String conversationId, ListMessage messages) { String key CONVERSATION_KEY_PREFIX conversationId; for (Message msg : messages) { redisTemplate.opsForList().rightPush(key, msg); } redisTemplate.expire(key, TTL); } Override public ListMessage get(String conversationId, int lastN) { String key CONVERSATION_KEY_PREFIX conversationId; Long size redisTemplate.opsForList().size(key); if (size null || size 0) { return List.of(); } long start Math.max(0, size - lastN); ListObject rawMessages redisTemplate.opsForList().range(key, start, -1); if (rawMessages null) { return List.of(); } return rawMessages.stream() .filter(Message.class::isInstance) .map(Message.class::cast) .toList(); } Override public void clear(String conversationId) { redisTemplate.delete(CONVERSATION_KEY_PREFIX conversationId); } }客户端集成与参数配置在业务 Service 中通过 Fluent API 将ChatClient配置好滑动窗口拦截器package com.architect.ai.service; import com.architect.ai.memory.advisor.SlidingWindowChatMemoryAdvisor; import org.springframework.ai.chat.client.ChatClient; import org.springframework.stereotype.Service; Service public class AssistantService { private final ChatClient chatClient; public AssistantService(ChatClient.Builder builder, SlidingWindowChatMemoryAdvisor slidingWindowAdvisor) { this.chatClient builder .defaultAdvisors(slidingWindowAdvisor) .defaultSystem(你是一名资深企业级架构顾问必须以严谨、精准的技术口吻回答问题。) .build(); } public String talk(String conversationId, String userInput) { return chatClient.prompt() .user(userInput) .advisors(advisorSpec - advisorSpec.param(chat_memory_conversation_id, conversationId)) .call() .content(); } }生产运行效果与基准测试在 100 轮连续高强度问答的压测场景下对比“全量历史堆叠”与“滑动窗口截断”的核心指标指标项全量历史追加未截断3000 Token 滑动窗口截断优化幅度平均第 50 轮 Prompt Token7,420 tokens2,890 tokens-61.0%P99 首字返回延迟 (TTFT)2,450 ms620 ms-74.7%100 轮总花费 (USD / 1k 次对话)$48.60$12.30-74.6%超限错误率 (400 error)18.3%0.0%完全消除通过在接入层引入 Token 级别的精确估算与动态窗口截断系统在保证多轮对话核心记忆连贯的同时彻底消除了上下文溢出导致的硬报错并显著压降了 LLM 运行成本。