尧图网站建设 
Agno 可靠性评估Reliability Eval实战指南验证 Agent 与 Team 的工具调用是否按预期执行【免费下载链接】agnoBuild, run, and manage agent platforms.项目地址: https://gitcode.com/GitHub_Trending/ag/agno在 Agno 中构建 Agent 与 Team 时模型是否说到做到——即是否真的调用了我们期望的工具、并以正确的参数调用——直接决定了系统的稳定性与可信度。本文围绕 cookbook/09_evals/reliability 目录下的可靠性评估示例系统讲解 Agno 的ReliabilityEval评估框架如何校验单次工具调用、多次工具调用、参数级匹配如何评估 Team 的委派与搜索流程如何将评估结果持久化到 PostgreSQL以及如何以异步方式运行评估。读完本文你将掌握一套可复制、可接入 CI 的 Agent 工具调用可靠性验证方案。什么是 Reliability Eval可靠性评估Reliability Eval用于回答一个核心问题模型是否做出了预期的工具调用expected tool calls。它不评判答案的语义质量而是聚焦调用行为的正确性——工具名是否匹配、参数是否精确、是否调用了预期之外的工具。以 ReliabilityEval 源码 为据评估器通过对比agent_response或team_response中实际发生的工具执行与expected_tool_calls声明输出一个结构化的 ReliabilityResult其中包含字段含义eval_status评估结论PASSED或FAILEDpassed_tool_calls实际执行且匹配预期的工具调用列表failed_tool_calls执行了预期之外的、在严格模式下被视为失败的工具调用missing_tool_calls声明了预期但未产生干净执行缺失、被拒或出错的工具additional_tool_callsallow_additional_tool_callsTrue时记录的额外合法调用failed_argument_checks/passed_argument_checks参数级校验的通过/失败结果关键设计从源码注释可确认从 2.8.0 起评估依据从消息侧请求requests升级为执行侧证据executions。一个预期工具只有在存在干净的 ToolExecution即tool_call_error不为真、未被暂停时才视为满足被tool_call_limit拒绝、执行报错或参数非法的调用即使曾在消息中出现过也不再让评估通过。历史轮次注入的消息from_history标记会被排除避免昨天的调用影响今天的严格评估。ReliabilityResult.assert_passed()通过assert self.eval_status PASSED将失败直接转化为断言异常因而可以天然嵌入 CI 流程。快速上手单工具调用的可靠性校验最简单也最常见的场景是Agent 只被期望调用一个工具例如使用CalculatorTools计算阶乘。参考 single_tool_calls/calculator.pyfrom typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools def factorial(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10! (ten factorial)?) evaluation ReliabilityEval( nameTool Call Reliability, agent_responseresponse, expected_tool_calls[factorial], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed() if __name__ __main__: factorial()代码流程清晰可见创建 Agent → 运行得到RunOutput→ 构造ReliabilityEval→run()执行评估 →assert_passed()判定。其中expected_tool_calls[factorial]与 CalculatorTools.factorial 的方法名一一对应是匹配的事实基准print_resultsTrue会在终端用 Rich 渲染一张 Reliability Summary 表格来自 ReliabilityResult.print_eval若结果缺失例如模型直接给答案而没调工具missing_tool_calls将包含factorial评估状态为FAILEDassert_passed()抛出断言。这与单元测试 test_reliability_eval.py 中test_exact_match_passes、test_fails_on_missing_expected_tool、test_fails_when_no_tools_called等用例验证的语义完全一致预期工具未产生干净执行即判失败。参数级校验不仅调对工具还要传对参数仅校验工具名往往不够。当业务依赖精确参数时需要使用expected_tool_call_arguments做参数匹配。同一文件中的第二个示例def multiply_with_argument_check(): Verify that the tool was called with the correct arguments. agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10 * 5?) evaluation ReliabilityEval( nameTool Call Argument Validation, agent_responseresponse, expected_tool_calls[multiply], expected_tool_call_arguments{ multiply: {a: 10, b: 5}, }, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()参数声明的两种形态源码字段定义单次校验{multiply: {a: 10, b: 5}}—— 期望至少有一次multiply调用的参数同时满足a 10且b 5多次校验{add: [{a: 2, b: 2}, {a: 3, b: 3}]}—— 列表中的每个 spec 都必须被至少一次干净调用命中才判定通过。从 参数匹配实现 看参数取自ToolExecution.tool_args已解析的 JSON 参数空参数归一化为{}且只从干净执行中收集——消息侧请求即使携带参数若调用本身未真正工作也不能满足参数检查。若某个工具已被判为缺失missing_names其参数检查会被自动跳过避免重复报错。多工具调用与子集匹配真实任务往往需要一连串工具协同。参考 multiple_tool_calls/calculator.py严格模式所有预期工具都必须出现def multiply_and_exponentiate(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run( What is 10*5 then to the power of 2? do it step by step ) evaluation ReliabilityEval( nameTool Calls Reliability, agent_responseresponse, expected_tool_calls[multiply, exponentiate], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()默认allow_additional_tool_callsFalse语义是精确匹配exact matchmultiply、exponentiate都必须有干净执行与此同时任何不在预期列表内的工具调用都会被记入failed_tool_calls。单元测试 test_exact_match_fails_on_unexpected_tool 印证预期[multiply]而实际多调用了exponentiate评估即失败。宽松模式子集匹配subset matchingdef subset_matching(): Only require multiply -- extra tool calls like exponentiate are allowed. agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run( What is 10*5 then to the power of 2? do it step by step ) evaluation ReliabilityEval( nameSubset Tool Calls, agent_responseresponse, expected_tool_calls[multiply], allow_additional_tool_callsTrue, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()当allow_additional_tool_callsTrue时评估退化为子集匹配只要求multiply存在exponentiate等额外调用被记入additional_tool_calls仅作记录不判失败。这在模型多走一步但核心行为正确的场景下非常实用——例如你只关心 Agent 是否完成了主工具调用而不强求它的推理路径完全固定。团队可靠性校验 Team 的委派与搜索流程可靠性评估同样适用于 Team。参考 team/ai_news.py一个 News Searcher 成员 Agent 负责调用WebSearchTools(enable_newsTrue)搜索新闻外层 Team 负责委派任务。from typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.team import TeamRunOutput from agno.team.team import Team from agno.tools.websearch import WebSearchTools team_member Agent( nameNews Searcher, modelOpenAIChat(gpt-5.6-luna), roleSearches the web for the latest news., tools[WebSearchTools(enable_newsTrue)], ) team Team( nameNews Research Team, modelOpenAIChat(gpt-5.6-luna), members[team_member], markdownTrue, show_members_responsesTrue, ) expected_tool_calls [ delegate_task_to_member, search_news, ] def evaluate_team_reliability(): response: TeamRunOutput team.run(What is the latest news on AI?) evaluation ReliabilityEval( nameTeam Reliability Evaluation, team_responseresponse, expected_tool_callsexpected_tool_calls, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed() if __name__ __main__: evaluate_team_reliability()这里有两个关键差异点传入team_response类型TeamRunOutput而非agent_response。ReliabilityEval.run()明确要求二者必须且只能提供一个参数校验逻辑证据来自嵌套成员响应。Team 的工具调用发生在成员 Agent 上且成员本身也可能是 Team。从 _collect_member_evidence 实现 可以确认评估器会递归遍历member_responses把每一层嵌套的tools与messages汇总后统一匹配。因此expected_tool_calls可以同时包含 Team 层级的delegate_task_to_member和成员层级的search_news二者都必须有干净执行。评估结果持久化写入 PostgreSQL生产环境中评估结果需要留痕、可追溯。参考 db_logging.py评估器内置数据库写入能力from typing import Optional from agno.agent import Agent from agno.db.postgres.postgres import PostgresDb from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools # Create Database db_url postgresqlpsycopg://ai:ailocalhost:5432/ai db PostgresDb(db_urldb_url, eval_tableeval_runs) # Create Agent agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) if __name__ __main__: response: RunOutput agent.run(What is 10!?) evaluation ReliabilityEval( dbdb, nameTool Call Reliability, agent_responseresponse, expected_tool_calls[factorial], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()通过PostgresDb(db_url..., eval_tableeval_runs)指定连接串与评估结果表评估通过EvalType.RELIABILITY类型将run_id、run_data即ReliabilityResult的完整序列化、评估输入expected_tool_calls、allow_additional_tool_calls、expected_tool_call_arguments以及agent_id/team_id/model_id/model_provider一并写入数据库写入逻辑同一套持久化机制也支持file_path_to_save_results参数支持{name}、{run_id}占位符可将结果保存到本地文件。注意若使用异步数据库AsyncBaseDbrun()会抛出ValueError必须改用arun()源码约束。异步评估arun()与并发编排需要将多个评估并行编排、或接入异步 Agent 平台时使用异步接口。参考 reliability_async.pyimport asyncio from typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools def factorial(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10!?) evaluation ReliabilityEval( agent_responseresponse, expected_tool_calls[factorial], ) # Run the evaluation calling the arun method. result: Optional[ReliabilityResult] asyncio.run( evaluation.arun(print_resultsTrue) ) if result: result.assert_passed() if __name__ __main__: factorial()arun(print_resultsTrue)与run()具备对等的完整链路生成run_id→ 富文本 spinner →_evaluate核心判定 → 可选落盘 → 可选打印 → 通过async_log_eval写入异步数据库 →async_log_eval_telemetry上报遥测异步实现。在异步 Agent 循环中可直接await evaluation.arun(...)而无需包一层asyncio.run。将可靠性评估接入 CI综合以上能力一套推荐的接入模式是对每个关键 Agent/Team编写独立评估函数最后统一断言。失败即抛出AssertionError退出码非零CI 自然红灯python single_tool_calls/calculator.py python multiple_tool_calls/calculator.py python team/ai_news.py python db_logging.py # 需本地 PostgreSQL 可用调试小贴士eval_status为FAILED时优先查看missing_tool_calls——若条目带有(requested but refused/errored — execution matching, new in 2.8.0)注释说明模型发起了调用但执行被拒/出错问题在运行环境或tool_call_limit配置而非评估器本身failed_tool_calls出现预期外工具时检查是否应设置allow_additional_tool_callsTrue或审视提示词是否约束不足期望值写错例如工具名与 CalculatorTools 中实际的multiply/exponentiate/factorial不一致会导致误报务必以工具类中的真实方法名为准不需要终端输出的自动化场景可设show_spinnerFalse关闭进度动画对应测试 test_show_spinner_disabled。小结Reliability Eval 是 Agno 评估体系cookbook/09_evals中专注行为正确性的一环。通过 ReliabilityEval 的expected_tool_calls、expected_tool_call_arguments、allow_additional_tool_calls三个核心配置你可以分别验证工具是否被调用、参数是否精确、额外调用是否被容忍通过agent_response/team_response双通道覆盖单 Agent 与嵌套 Team通过db、file_path_to_save_results与run/arun双接口满足持久化与异步编排需求。将其与 CI 结合即可为每个模型迭代建立可回归的工具调用护栏。【免费下载链接】agnoBuild, run, and manage agent platforms.项目地址: https://gitcode.com/GitHub_Trending/ag/agno创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 Agno 可靠性评估（Reliability Eval）实战指南：验证 Agent 与 Team 的工具调用是否按预期执行
Agno 可靠性评估Reliability Eval实战指南验证 Agent 与 Team 的工具调用是否按预期执行【免费下载链接】agnoBuild, run, and manage agent platforms.项目地址: https://gitcode.com/GitHub_Trending/ag/agno在 Agno 中构建 Agent 与 Team 时模型是否说到做到——即是否真的调用了我们期望的工具、并以正确的参数调用——直接决定了系统的稳定性与可信度。本文围绕 cookbook/09_evals/reliability 目录下的可靠性评估示例系统讲解 Agno 的ReliabilityEval评估框架如何校验单次工具调用、多次工具调用、参数级匹配如何评估 Team 的委派与搜索流程如何将评估结果持久化到 PostgreSQL以及如何以异步方式运行评估。读完本文你将掌握一套可复制、可接入 CI 的 Agent 工具调用可靠性验证方案。什么是 Reliability Eval可靠性评估Reliability Eval用于回答一个核心问题模型是否做出了预期的工具调用expected tool calls。它不评判答案的语义质量而是聚焦调用行为的正确性——工具名是否匹配、参数是否精确、是否调用了预期之外的工具。以 ReliabilityEval 源码 为据评估器通过对比agent_response或team_response中实际发生的工具执行与expected_tool_calls声明输出一个结构化的 ReliabilityResult其中包含字段含义eval_status评估结论PASSED或FAILEDpassed_tool_calls实际执行且匹配预期的工具调用列表failed_tool_calls执行了预期之外的、在严格模式下被视为失败的工具调用missing_tool_calls声明了预期但未产生干净执行缺失、被拒或出错的工具additional_tool_callsallow_additional_tool_callsTrue时记录的额外合法调用failed_argument_checks/passed_argument_checks参数级校验的通过/失败结果关键设计从源码注释可确认从 2.8.0 起评估依据从消息侧请求requests升级为执行侧证据executions。一个预期工具只有在存在干净的 ToolExecution即tool_call_error不为真、未被暂停时才视为满足被tool_call_limit拒绝、执行报错或参数非法的调用即使曾在消息中出现过也不再让评估通过。历史轮次注入的消息from_history标记会被排除避免昨天的调用影响今天的严格评估。ReliabilityResult.assert_passed()通过assert self.eval_status PASSED将失败直接转化为断言异常因而可以天然嵌入 CI 流程。快速上手单工具调用的可靠性校验最简单也最常见的场景是Agent 只被期望调用一个工具例如使用CalculatorTools计算阶乘。参考 single_tool_calls/calculator.pyfrom typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools def factorial(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10! (ten factorial)?) evaluation ReliabilityEval( nameTool Call Reliability, agent_responseresponse, expected_tool_calls[factorial], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed() if __name__ __main__: factorial()代码流程清晰可见创建 Agent → 运行得到RunOutput→ 构造ReliabilityEval→run()执行评估 →assert_passed()判定。其中expected_tool_calls[factorial]与 CalculatorTools.factorial 的方法名一一对应是匹配的事实基准print_resultsTrue会在终端用 Rich 渲染一张 Reliability Summary 表格来自 ReliabilityResult.print_eval若结果缺失例如模型直接给答案而没调工具missing_tool_calls将包含factorial评估状态为FAILEDassert_passed()抛出断言。这与单元测试 test_reliability_eval.py 中test_exact_match_passes、test_fails_on_missing_expected_tool、test_fails_when_no_tools_called等用例验证的语义完全一致预期工具未产生干净执行即判失败。参数级校验不仅调对工具还要传对参数仅校验工具名往往不够。当业务依赖精确参数时需要使用expected_tool_call_arguments做参数匹配。同一文件中的第二个示例def multiply_with_argument_check(): Verify that the tool was called with the correct arguments. agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10 * 5?) evaluation ReliabilityEval( nameTool Call Argument Validation, agent_responseresponse, expected_tool_calls[multiply], expected_tool_call_arguments{ multiply: {a: 10, b: 5}, }, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()参数声明的两种形态源码字段定义单次校验{multiply: {a: 10, b: 5}}—— 期望至少有一次multiply调用的参数同时满足a 10且b 5多次校验{add: [{a: 2, b: 2}, {a: 3, b: 3}]}—— 列表中的每个 spec 都必须被至少一次干净调用命中才判定通过。从 参数匹配实现 看参数取自ToolExecution.tool_args已解析的 JSON 参数空参数归一化为{}且只从干净执行中收集——消息侧请求即使携带参数若调用本身未真正工作也不能满足参数检查。若某个工具已被判为缺失missing_names其参数检查会被自动跳过避免重复报错。多工具调用与子集匹配真实任务往往需要一连串工具协同。参考 multiple_tool_calls/calculator.py严格模式所有预期工具都必须出现def multiply_and_exponentiate(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run( What is 10*5 then to the power of 2? do it step by step ) evaluation ReliabilityEval( nameTool Calls Reliability, agent_responseresponse, expected_tool_calls[multiply, exponentiate], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()默认allow_additional_tool_callsFalse语义是精确匹配exact matchmultiply、exponentiate都必须有干净执行与此同时任何不在预期列表内的工具调用都会被记入failed_tool_calls。单元测试 test_exact_match_fails_on_unexpected_tool 印证预期[multiply]而实际多调用了exponentiate评估即失败。宽松模式子集匹配subset matchingdef subset_matching(): Only require multiply -- extra tool calls like exponentiate are allowed. agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run( What is 10*5 then to the power of 2? do it step by step ) evaluation ReliabilityEval( nameSubset Tool Calls, agent_responseresponse, expected_tool_calls[multiply], allow_additional_tool_callsTrue, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()当allow_additional_tool_callsTrue时评估退化为子集匹配只要求multiply存在exponentiate等额外调用被记入additional_tool_calls仅作记录不判失败。这在模型多走一步但核心行为正确的场景下非常实用——例如你只关心 Agent 是否完成了主工具调用而不强求它的推理路径完全固定。团队可靠性校验 Team 的委派与搜索流程可靠性评估同样适用于 Team。参考 team/ai_news.py一个 News Searcher 成员 Agent 负责调用WebSearchTools(enable_newsTrue)搜索新闻外层 Team 负责委派任务。from typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.team import TeamRunOutput from agno.team.team import Team from agno.tools.websearch import WebSearchTools team_member Agent( nameNews Searcher, modelOpenAIChat(gpt-5.6-luna), roleSearches the web for the latest news., tools[WebSearchTools(enable_newsTrue)], ) team Team( nameNews Research Team, modelOpenAIChat(gpt-5.6-luna), members[team_member], markdownTrue, show_members_responsesTrue, ) expected_tool_calls [ delegate_task_to_member, search_news, ] def evaluate_team_reliability(): response: TeamRunOutput team.run(What is the latest news on AI?) evaluation ReliabilityEval( nameTeam Reliability Evaluation, team_responseresponse, expected_tool_callsexpected_tool_calls, ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed() if __name__ __main__: evaluate_team_reliability()这里有两个关键差异点传入team_response类型TeamRunOutput而非agent_response。ReliabilityEval.run()明确要求二者必须且只能提供一个参数校验逻辑证据来自嵌套成员响应。Team 的工具调用发生在成员 Agent 上且成员本身也可能是 Team。从 _collect_member_evidence 实现 可以确认评估器会递归遍历member_responses把每一层嵌套的tools与messages汇总后统一匹配。因此expected_tool_calls可以同时包含 Team 层级的delegate_task_to_member和成员层级的search_news二者都必须有干净执行。评估结果持久化写入 PostgreSQL生产环境中评估结果需要留痕、可追溯。参考 db_logging.py评估器内置数据库写入能力from typing import Optional from agno.agent import Agent from agno.db.postgres.postgres import PostgresDb from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools # Create Database db_url postgresqlpsycopg://ai:ailocalhost:5432/ai db PostgresDb(db_urldb_url, eval_tableeval_runs) # Create Agent agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) if __name__ __main__: response: RunOutput agent.run(What is 10!?) evaluation ReliabilityEval( dbdb, nameTool Call Reliability, agent_responseresponse, expected_tool_calls[factorial], ) result: Optional[ReliabilityResult] evaluation.run(print_resultsTrue) if result: result.assert_passed()通过PostgresDb(db_url..., eval_tableeval_runs)指定连接串与评估结果表评估通过EvalType.RELIABILITY类型将run_id、run_data即ReliabilityResult的完整序列化、评估输入expected_tool_calls、allow_additional_tool_calls、expected_tool_call_arguments以及agent_id/team_id/model_id/model_provider一并写入数据库写入逻辑同一套持久化机制也支持file_path_to_save_results参数支持{name}、{run_id}占位符可将结果保存到本地文件。注意若使用异步数据库AsyncBaseDbrun()会抛出ValueError必须改用arun()源码约束。异步评估arun()与并发编排需要将多个评估并行编排、或接入异步 Agent 平台时使用异步接口。参考 reliability_async.pyimport asyncio from typing import Optional from agno.agent import Agent from agno.eval.reliability import ReliabilityEval, ReliabilityResult from agno.models.openai import OpenAIChat from agno.run.agent import RunOutput from agno.tools.calculator import CalculatorTools def factorial(): agent Agent( modelOpenAIChat(idgpt-5.2), tools[CalculatorTools()], ) response: RunOutput agent.run(What is 10!?) evaluation ReliabilityEval( agent_responseresponse, expected_tool_calls[factorial], ) # Run the evaluation calling the arun method. result: Optional[ReliabilityResult] asyncio.run( evaluation.arun(print_resultsTrue) ) if result: result.assert_passed() if __name__ __main__: factorial()arun(print_resultsTrue)与run()具备对等的完整链路生成run_id→ 富文本 spinner →_evaluate核心判定 → 可选落盘 → 可选打印 → 通过async_log_eval写入异步数据库 →async_log_eval_telemetry上报遥测异步实现。在异步 Agent 循环中可直接await evaluation.arun(...)而无需包一层asyncio.run。将可靠性评估接入 CI综合以上能力一套推荐的接入模式是对每个关键 Agent/Team编写独立评估函数最后统一断言。失败即抛出AssertionError退出码非零CI 自然红灯python single_tool_calls/calculator.py python multiple_tool_calls/calculator.py python team/ai_news.py python db_logging.py # 需本地 PostgreSQL 可用调试小贴士eval_status为FAILED时优先查看missing_tool_calls——若条目带有(requested but refused/errored — execution matching, new in 2.8.0)注释说明模型发起了调用但执行被拒/出错问题在运行环境或tool_call_limit配置而非评估器本身failed_tool_calls出现预期外工具时检查是否应设置allow_additional_tool_callsTrue或审视提示词是否约束不足期望值写错例如工具名与 CalculatorTools 中实际的multiply/exponentiate/factorial不一致会导致误报务必以工具类中的真实方法名为准不需要终端输出的自动化场景可设show_spinnerFalse关闭进度动画对应测试 test_show_spinner_disabled。小结Reliability Eval 是 Agno 评估体系cookbook/09_evals中专注行为正确性的一环。通过 ReliabilityEval 的expected_tool_calls、expected_tool_call_arguments、allow_additional_tool_calls三个核心配置你可以分别验证工具是否被调用、参数是否精确、额外调用是否被容忍通过agent_response/team_response双通道覆盖单 Agent 与嵌套 Team通过db、file_path_to_save_results与run/arun双接口满足持久化与异步编排需求。将其与 CI 结合即可为每个模型迭代建立可回归的工具调用护栏。【免费下载链接】agnoBuild, run, and manage agent platforms.项目地址: https://gitcode.com/GitHub_Trending/ag/agno创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考