尧图网站建设 
trace-to-training-data 转换配方指南将评估轨迹转换为 SFT 样本与 DPO 偏好对【免费下载链接】agentsMulti-harness agentic plugin marketplace for Claude Code, Codex, Cursor, OpenCode, GitHub Copilot, and Google Antigravity项目地址: https://gitcode.com/GitHub_Trending/agents24/agents本篇文章以 GitHub 推荐项目精选 / agents24 仓库中llm-finetuning插件trace-to-training-dataskill 的 references/conversion-recipes.md 为核心系统讲解五类 JSONL→JSONL 转换配方带评分轨迹转 SFT 样本、双轨迹构造 DPO 偏好对、人工纠错记录转 SFT 样本、拒绝采样循环以及任何转换批次合入训练集前必须执行的 goldens 留出校验。读完本文你将掌握如何把eval-harness-first产出的分级轨迹runs/run-id/results.json精确转换为dataset-curation定义的目标格式字段名逐字对齐而非意译并理解每类转换背后的筛选原则top-reward 分位、μ−2σ 拒绝成员选择、holdout 防泄漏及其仓库源码依据。前置背景转换发生在评分之后trace-to-training-data的定位非常明确转换发生在评分之后。正如其 SKILL.md 开篇所述本 skill 假设eval-harness-first已经完成了对轨迹的评分——goldens、graders 以及runs/run-id/results.json在转换开始前就已存在。这也是 eval-harness-first 所称flywheel飞轮的闭环边缘the same labeled traces become the training set同一批已标注轨迹变成训练集。输入eval/goldens.jsonl加runs/run-id/results.json每行携带task_id、来自 grader 的verdict以及任务支持标量分数时的rewardjudge 分数、执行部分得分或 RLVR verifier 分数。输出与dataset-curation的 Format Selection 表完全一致的格式——SFT 的messages行或 DPO 的prompt/chosen/rejected对——因此本 skill 的输出就是下游 skill 的输入中间无需任何重塑步骤。eval-harness-first的目录契约SKILL.md规定了results.json的标准位置每个 checkpoint 的每次运行结果都位于runs/run-id/results.json绝不能写到eval/下eval/是跨运行持久化的固定标尺runs/则是一次性产物。conversion-recipes.md 明确了两条硬性约束目标字段名逐字复制输出记录的字段名必须与dataset-curation的 Format Selection 表完全一致copied, not paraphrased。不出现任何基础模型名文档中所有示例使用占位符确需指定模型时从finetuning-method-selection的 references/model-catalog.md 加载。配方 1带评分轨迹转 SFT 样本这是最基础的转换一条通过了评分阈值高于本批次 top-fraction 阈值的单轮轨迹转成一条 SFT 训练行。输入runs/run-id/results.json中的一行{task_id: t-118, trace_id: t-118-a1, messages: [{role: user, content: Summarize the incident report in two sentences.}, {role: assistant, content: A misconfigured retry policy caused a 12-minute outage in the billing service. Root cause was fixed and a regression test was added.}], verdict: pass, reward: 0.94, grader: rubric_judge}输出dataset-curation的 ChatML 形态评分元数据被剥离因为训练器只需要messages{messages: [{role: user, content: Summarize the incident report in two sentences.}, {role: assistant, content: A misconfigured retry policy caused a 12-minute outage in the billing service. Root cause was fixed and a regression test was added.}]}转换后只有messages存活。task_id和trace_id仍会写入数据集卡的 provenance 字段见配方 5只是从训练行本身剥离——不是彻底丢弃。这正对应dataset-curation数据集卡六字段中的ProvenanceSKILL.md每一行都必须能回溯到源run_id和trace_id无源可溯的行不允许合入。输出格式对应的完整 ChatML 形态可参考 references/formats-and-templates.mdmessages列表每行一条是SFTTrainer原生模板化与 loss 掩码的形状。配方 2两条带评分轨迹构造 DPO 偏好对DPO 偏好对必须来自同一task_id的两条轨迹同一 rollout 批次、奖励不同——绝不跨任务取最好 vs 最坏轨迹否则模型学到的是偏好任务 A 而非任务 B而不是偏好某个回答。这是trace-to-training-data的 SKILL.md Preference Pairs From Traces 节的核心原则。输入两条共享task_id的轨迹{task_id: t-204, trace_id: t-204-a1, messages: [{role: user, content: Write a commit message for a null-check fix.}, {role: assistant, content: Fix null pointer exception in user lookup by validating the session before dereferencing it.}], verdict: pass, reward: 0.88, grader: rubric_judge} {task_id: t-204, trace_id: t-204-a4, messages: [{role: user, content: Write a commit message for a null-check fix.}, {role: assistant, content: misc changes}], verdict: fail, reward: 0.11, grader: rubric_judge}选择逻辑依据preference-optimization的 Pair Construction 公式——chosen是该task_id下奖励最高的轨迹rejected则是该任务轨迹集中最接近奖励分布 μ−2σ的那条而非默认取最低奖励的轨迹。本示例只有两条候选低分轨迹恰好就是 μ−2σ 的选取结果当每个任务采样的候选数更多时rejected 会选在最小值之上的成员。def select_pair(trajectories): trajectories: same task_id, each a dict with reward and messages. Returns (chosen, rejected) — always two distinct records; raises if fewer than two trajectories are given. if len(trajectories) 2: raise ValueError(select_pair needs 2 trajectories to form a pair) ranked sorted(trajectories, keylambda t: t[reward]) chosen ranked[-1] candidates [t for t in trajectories if t is not chosen] rewards [t[reward] for t in trajectories] mu sum(rewards) / len(rewards) variance sum((r - mu) ** 2 for r in rewards) / len(rewards) sigma variance ** 0.5 target mu - 2 * sigma rejected min(candidates, keylambda t: abs(t[reward] - target)) return chosen, rejected输出dataset-curation的 DPO 对形态prompt取自共享的 user 轮chosen/rejected分别取自两条轨迹的最后一轮 assistant 回复{prompt: Write a commit message for a null-check fix., chosen: Fix null pointer exception in user lookup by validating the session before dereferencing it., rejected: misc changes}为何 μ−2σ 而非绝对最小值preference-optimization的 SKILL.md Pair Construction 节给出了依据naive best-vs-worst 构造max vs 绝对最小值会随规模增大而退化μ−2σ 选择对同一规模敏感性更稳健。这也与 SKILL.md 中judge-scored delta selection互补——先构建完整候选集再用 chosen-minus-rejected 的 judge 分数差筛选最高 delta 子集而不是前置截断生成量。配方 3人工纠错记录转 SFT 样本当一条失败轨迹经过人类专家修正为正确输出时该修正直接成为黄金 SFT 样本Langfuse pattern无需奖励阈值——人类已经完成了验证。因此输入记录可以不带reward字段。输入{task_id: t-311, trace_id: t-311-a2, messages: [{role: user, content: Extract the invoice total as a JSON number.}, {role: assistant, content: The total is around $4,200}], verdict: fail, grader: schema_compliance, correction: {content: {\total\: 4200.00}, corrected_by: reviewer-07}}输出——修正后的内容替换失败的 assistant 轮原始失败内容永远不会进入训练集{messages: [{role: user, content: Extract the invoice total as a JSON number.}, {role: assistant, content: {\total\: 4200.00}}]}依据trace-to-training-dataSKILL.md 的 SFT From Traces 节这类纠错行应直接路由进 SFT 集跳过配方 4 的奖励阈值门控。注意correction对象中还保留了corrected_by字段可作为溯源元数据写入数据集卡。配方 4拒绝采样循环拒绝采样rejection sampling是dataset-curationSKILL.md 点名的两大合成数据主力之一另一个是 Magpie对每个 prompt 采样多条候选补全逐条评分只保留奖励最高的 top 分位。这在 SKILL.md 中被称为 Agent-lightning pattern。MAX_CANDIDATES 32 # ceiling on model calls per prompt for this recipe def rejection_sample(prompt, policy, grader, n8, keep_fraction0.25): Generate n candidates for prompt, grade each, and keep the top keep_fraction by reward. This recipe is for small fixed batches — n is capped at MAX_CANDIDATES; a larger sampling budget needs a dedicated rollout pipeline with its own concurrency and cost controls, not this loop. if n MAX_CANDIDATES: raise ValueError(fn{n} exceeds MAX_CANDIDATES{MAX_CANDIDATES}) candidates [policy.generate(prompt) for _ in range(n)] graded [(c, grader.score(prompt, c)) for c in candidates] graded.sort(keylambda pair: pair[1], reverseTrue) keep_n max(1, int(len(graded) * keep_fraction)) kept graded[:keep_n] return [ {messages: [ {role: user, content: prompt}, {role: assistant, content: completion}, ]} for completion, reward in kept ]参数要点与边界条件MAX_CANDIDATES 32是本配方的每 prompt 模型调用上限超出即抛ValueError——本配方只面向小型固定批次更大的采样预算需要专门的 rollout 流水线自带并发与成本控制而不是这个循环。keep_n max(1, int(len(graded) * keep_fraction))保证至少保留 1 条防止极端分布下出现空集。调参原则以keep_fraction0.25、n8为例每个 prompt 约有 2 条候选进入 SFT 集。文档明确提示应针对批次奖励分布调整keep_fraction而非固定数量——因为更难的 prompt 集会整体下移奖励分布固定数量会导致筛选失效。dataset-curation的合成数据经验可作为本配方的配套参考典型过滤后接受率为 10–30%规划体量时需按此放大原始生成量例如 10,000 行目标、15% 接受率需约 65,000 条原始生成。配方 5Goldens 留出校验合入前的强制关卡任何转换批次合入训练集之前都必须执行此校验。依据trace-to-training-dataSKILL.md 的 Hygiene 节golden ID 泄漏进训练数据会在后续每一次针对同一 golden 的评测中静默抬高分数——模型在评测题目本身上训练过。import json def load_golden_ids(goldens_path): with open(goldens_path) as f: return {json.loads(line)[task_id] for line in f} def filter_holdout(candidate_rows, golden_ids): candidate_rows: dicts still carrying task_id before provenance stripping. Returns only rows whose task_id never appears in the goldens. kept, dropped [], [] for row in candidate_rows: if row[task_id] in golden_ids: dropped.append(row) else: kept.append(row) return kept, dropped三个关键操作纪律执行时机必须在配方 1 所示的messages-only 剥离之前运行filter_holdout——一旦task_id被剥离校验就没有可匹配的字段了。记录而非静默丢弃必须记录dropped而非悄悄丢弃大量dropped通常意味着轨迹采集环节在重复采样 goldens而非生产流量——这是采集管线故障的信号。与 eval 契约的关系eval-harness-first的 flywheel 步骤明确要求Everyeval/goldens.jsonlID stays excluded from training data by IDgoldens 同时充当 CI 回归套件像代码一样版本化、评审时 diff、按 release 打 tag。本校验正是这一契约在数据侧的落地执行。转换后的下一步合入前需要过的关卡conversion-recipes.md 的五个配方产出的 SFT/DPO 行在真正进入训练前还要满足dataset-curation的 Phase 2 退出清单SKILL.md格式匹配目标方法上文的 Format Selection 表模板在拼接packing之前应用loss 只掩码到 assistant 轮解码并人工阅读 5–10 条 packed 序列最终混合中真实数据占比 ≥25%数据集卡六字段齐全——Provenance可回溯到trace-to-training-data输出、Counts、合成/真实比例、去重方法、所用模板、packing 配置。此外 Hygiene 节还要求任何行合入前需运行密钥/PII 扫描生产日志来源的轨迹可能携带凭据、API key、token 或客户数据敏感字段在脱敏后仍残留则 fail-closed 丢弃该行并针对已有训练集做去重精确匹配或 embedding 相似度匹配dataset-curation的 dedup method 字段——不只是对新转换行内部去重。preference-optimization的 references/method-configs.md 提供了 DPO 训练消费这些偏好对的具体配置模板DPOConfig使用beta0.1、learning_rate7e-75e-7–1e-6 区间低于产出该 checkpoint 的 SFT 学习率、1–2 epochref_modelNone表示 TRL 从model派生冻结参考模型迭代 on-policy DPO 的后续轮次则显式传入上一轮 checkpoint 作为ref_model。五个配方一览配方输入输出形态关键筛选原则1. Graded Trace → SFT单条通过阈值、带reward的轨迹messagesChatML保留 top-reward 分位非所有通过者2. 双轨迹 → DPO 对同一task_id的两条轨迹prompt/chosen/rejectedchosen最高分rejectedμ−2σ 最近者非最小值3. 纠错记录 → SFT失败轨迹 人类修正messages替换失败 assistant 轮人类已验证跳过奖励阈值4. 拒绝采样循环同一 prompt 的 n 条候选messages保留 topkeep_fractionn≤32按奖励分布调keep_fraction5. Goldens 留出校验待合入候选行 goldens.jsonl过滤后的行保留task_id期间执行golden ID 泄漏会静默抬高评测分数延伸阅读均位于本仓库plugins/llm-finetuning插件下本 skill 主文档skills/trace-to-training-data/SKILL.md——SFT From Traces、Preference Pairs From Traces、Hygiene 三节的完整原则阐述上游评分引擎skills/eval-harness-first/SKILL.md——runs/run-id/results.json与eval/goldens.jsonl的目录契约下游目标格式skills/dataset-curation/references/formats-and-templates.md——ChatML、DPO 对、KTO、GRPO 各格式的完整 JSONL 示例偏好优化消费端skills/preference-optimization/SKILL.md 及 references/method-configs.md——μ−2σ 公式归属地与 DPO/ORPO/KTO/SimPO 配置基础模型选择skills/finetuning-method-selection/references/model-catalog.md——本插件唯一允许指定具体模型名的文件【免费下载链接】agentsMulti-harness agentic plugin marketplace for Claude Code, Codex, Cursor, OpenCode, GitHub Copilot, and Google Antigravity项目地址: https://gitcode.com/GitHub_Trending/agents24/agents创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 trace-to-training-data 转换配方指南：将评估轨迹转换为 SFT 样本与 DPO 偏好对
trace-to-training-data 转换配方指南将评估轨迹转换为 SFT 样本与 DPO 偏好对【免费下载链接】agentsMulti-harness agentic plugin marketplace for Claude Code, Codex, Cursor, OpenCode, GitHub Copilot, and Google Antigravity项目地址: https://gitcode.com/GitHub_Trending/agents24/agents本篇文章以 GitHub 推荐项目精选 / agents24 仓库中llm-finetuning插件trace-to-training-dataskill 的 references/conversion-recipes.md 为核心系统讲解五类 JSONL→JSONL 转换配方带评分轨迹转 SFT 样本、双轨迹构造 DPO 偏好对、人工纠错记录转 SFT 样本、拒绝采样循环以及任何转换批次合入训练集前必须执行的 goldens 留出校验。读完本文你将掌握如何把eval-harness-first产出的分级轨迹runs/run-id/results.json精确转换为dataset-curation定义的目标格式字段名逐字对齐而非意译并理解每类转换背后的筛选原则top-reward 分位、μ−2σ 拒绝成员选择、holdout 防泄漏及其仓库源码依据。前置背景转换发生在评分之后trace-to-training-data的定位非常明确转换发生在评分之后。正如其 SKILL.md 开篇所述本 skill 假设eval-harness-first已经完成了对轨迹的评分——goldens、graders 以及runs/run-id/results.json在转换开始前就已存在。这也是 eval-harness-first 所称flywheel飞轮的闭环边缘the same labeled traces become the training set同一批已标注轨迹变成训练集。输入eval/goldens.jsonl加runs/run-id/results.json每行携带task_id、来自 grader 的verdict以及任务支持标量分数时的rewardjudge 分数、执行部分得分或 RLVR verifier 分数。输出与dataset-curation的 Format Selection 表完全一致的格式——SFT 的messages行或 DPO 的prompt/chosen/rejected对——因此本 skill 的输出就是下游 skill 的输入中间无需任何重塑步骤。eval-harness-first的目录契约SKILL.md规定了results.json的标准位置每个 checkpoint 的每次运行结果都位于runs/run-id/results.json绝不能写到eval/下eval/是跨运行持久化的固定标尺runs/则是一次性产物。conversion-recipes.md 明确了两条硬性约束目标字段名逐字复制输出记录的字段名必须与dataset-curation的 Format Selection 表完全一致copied, not paraphrased。不出现任何基础模型名文档中所有示例使用占位符确需指定模型时从finetuning-method-selection的 references/model-catalog.md 加载。配方 1带评分轨迹转 SFT 样本这是最基础的转换一条通过了评分阈值高于本批次 top-fraction 阈值的单轮轨迹转成一条 SFT 训练行。输入runs/run-id/results.json中的一行{task_id: t-118, trace_id: t-118-a1, messages: [{role: user, content: Summarize the incident report in two sentences.}, {role: assistant, content: A misconfigured retry policy caused a 12-minute outage in the billing service. Root cause was fixed and a regression test was added.}], verdict: pass, reward: 0.94, grader: rubric_judge}输出dataset-curation的 ChatML 形态评分元数据被剥离因为训练器只需要messages{messages: [{role: user, content: Summarize the incident report in two sentences.}, {role: assistant, content: A misconfigured retry policy caused a 12-minute outage in the billing service. Root cause was fixed and a regression test was added.}]}转换后只有messages存活。task_id和trace_id仍会写入数据集卡的 provenance 字段见配方 5只是从训练行本身剥离——不是彻底丢弃。这正对应dataset-curation数据集卡六字段中的ProvenanceSKILL.md每一行都必须能回溯到源run_id和trace_id无源可溯的行不允许合入。输出格式对应的完整 ChatML 形态可参考 references/formats-and-templates.mdmessages列表每行一条是SFTTrainer原生模板化与 loss 掩码的形状。配方 2两条带评分轨迹构造 DPO 偏好对DPO 偏好对必须来自同一task_id的两条轨迹同一 rollout 批次、奖励不同——绝不跨任务取最好 vs 最坏轨迹否则模型学到的是偏好任务 A 而非任务 B而不是偏好某个回答。这是trace-to-training-data的 SKILL.md Preference Pairs From Traces 节的核心原则。输入两条共享task_id的轨迹{task_id: t-204, trace_id: t-204-a1, messages: [{role: user, content: Write a commit message for a null-check fix.}, {role: assistant, content: Fix null pointer exception in user lookup by validating the session before dereferencing it.}], verdict: pass, reward: 0.88, grader: rubric_judge} {task_id: t-204, trace_id: t-204-a4, messages: [{role: user, content: Write a commit message for a null-check fix.}, {role: assistant, content: misc changes}], verdict: fail, reward: 0.11, grader: rubric_judge}选择逻辑依据preference-optimization的 Pair Construction 公式——chosen是该task_id下奖励最高的轨迹rejected则是该任务轨迹集中最接近奖励分布 μ−2σ的那条而非默认取最低奖励的轨迹。本示例只有两条候选低分轨迹恰好就是 μ−2σ 的选取结果当每个任务采样的候选数更多时rejected 会选在最小值之上的成员。def select_pair(trajectories): trajectories: same task_id, each a dict with reward and messages. Returns (chosen, rejected) — always two distinct records; raises if fewer than two trajectories are given. if len(trajectories) 2: raise ValueError(select_pair needs 2 trajectories to form a pair) ranked sorted(trajectories, keylambda t: t[reward]) chosen ranked[-1] candidates [t for t in trajectories if t is not chosen] rewards [t[reward] for t in trajectories] mu sum(rewards) / len(rewards) variance sum((r - mu) ** 2 for r in rewards) / len(rewards) sigma variance ** 0.5 target mu - 2 * sigma rejected min(candidates, keylambda t: abs(t[reward] - target)) return chosen, rejected输出dataset-curation的 DPO 对形态prompt取自共享的 user 轮chosen/rejected分别取自两条轨迹的最后一轮 assistant 回复{prompt: Write a commit message for a null-check fix., chosen: Fix null pointer exception in user lookup by validating the session before dereferencing it., rejected: misc changes}为何 μ−2σ 而非绝对最小值preference-optimization的 SKILL.md Pair Construction 节给出了依据naive best-vs-worst 构造max vs 绝对最小值会随规模增大而退化μ−2σ 选择对同一规模敏感性更稳健。这也与 SKILL.md 中judge-scored delta selection互补——先构建完整候选集再用 chosen-minus-rejected 的 judge 分数差筛选最高 delta 子集而不是前置截断生成量。配方 3人工纠错记录转 SFT 样本当一条失败轨迹经过人类专家修正为正确输出时该修正直接成为黄金 SFT 样本Langfuse pattern无需奖励阈值——人类已经完成了验证。因此输入记录可以不带reward字段。输入{task_id: t-311, trace_id: t-311-a2, messages: [{role: user, content: Extract the invoice total as a JSON number.}, {role: assistant, content: The total is around $4,200}], verdict: fail, grader: schema_compliance, correction: {content: {\total\: 4200.00}, corrected_by: reviewer-07}}输出——修正后的内容替换失败的 assistant 轮原始失败内容永远不会进入训练集{messages: [{role: user, content: Extract the invoice total as a JSON number.}, {role: assistant, content: {\total\: 4200.00}}]}依据trace-to-training-dataSKILL.md 的 SFT From Traces 节这类纠错行应直接路由进 SFT 集跳过配方 4 的奖励阈值门控。注意correction对象中还保留了corrected_by字段可作为溯源元数据写入数据集卡。配方 4拒绝采样循环拒绝采样rejection sampling是dataset-curationSKILL.md 点名的两大合成数据主力之一另一个是 Magpie对每个 prompt 采样多条候选补全逐条评分只保留奖励最高的 top 分位。这在 SKILL.md 中被称为 Agent-lightning pattern。MAX_CANDIDATES 32 # ceiling on model calls per prompt for this recipe def rejection_sample(prompt, policy, grader, n8, keep_fraction0.25): Generate n candidates for prompt, grade each, and keep the top keep_fraction by reward. This recipe is for small fixed batches — n is capped at MAX_CANDIDATES; a larger sampling budget needs a dedicated rollout pipeline with its own concurrency and cost controls, not this loop. if n MAX_CANDIDATES: raise ValueError(fn{n} exceeds MAX_CANDIDATES{MAX_CANDIDATES}) candidates [policy.generate(prompt) for _ in range(n)] graded [(c, grader.score(prompt, c)) for c in candidates] graded.sort(keylambda pair: pair[1], reverseTrue) keep_n max(1, int(len(graded) * keep_fraction)) kept graded[:keep_n] return [ {messages: [ {role: user, content: prompt}, {role: assistant, content: completion}, ]} for completion, reward in kept ]参数要点与边界条件MAX_CANDIDATES 32是本配方的每 prompt 模型调用上限超出即抛ValueError——本配方只面向小型固定批次更大的采样预算需要专门的 rollout 流水线自带并发与成本控制而不是这个循环。keep_n max(1, int(len(graded) * keep_fraction))保证至少保留 1 条防止极端分布下出现空集。调参原则以keep_fraction0.25、n8为例每个 prompt 约有 2 条候选进入 SFT 集。文档明确提示应针对批次奖励分布调整keep_fraction而非固定数量——因为更难的 prompt 集会整体下移奖励分布固定数量会导致筛选失效。dataset-curation的合成数据经验可作为本配方的配套参考典型过滤后接受率为 10–30%规划体量时需按此放大原始生成量例如 10,000 行目标、15% 接受率需约 65,000 条原始生成。配方 5Goldens 留出校验合入前的强制关卡任何转换批次合入训练集之前都必须执行此校验。依据trace-to-training-dataSKILL.md 的 Hygiene 节golden ID 泄漏进训练数据会在后续每一次针对同一 golden 的评测中静默抬高分数——模型在评测题目本身上训练过。import json def load_golden_ids(goldens_path): with open(goldens_path) as f: return {json.loads(line)[task_id] for line in f} def filter_holdout(candidate_rows, golden_ids): candidate_rows: dicts still carrying task_id before provenance stripping. Returns only rows whose task_id never appears in the goldens. kept, dropped [], [] for row in candidate_rows: if row[task_id] in golden_ids: dropped.append(row) else: kept.append(row) return kept, dropped三个关键操作纪律执行时机必须在配方 1 所示的messages-only 剥离之前运行filter_holdout——一旦task_id被剥离校验就没有可匹配的字段了。记录而非静默丢弃必须记录dropped而非悄悄丢弃大量dropped通常意味着轨迹采集环节在重复采样 goldens而非生产流量——这是采集管线故障的信号。与 eval 契约的关系eval-harness-first的 flywheel 步骤明确要求Everyeval/goldens.jsonlID stays excluded from training data by IDgoldens 同时充当 CI 回归套件像代码一样版本化、评审时 diff、按 release 打 tag。本校验正是这一契约在数据侧的落地执行。转换后的下一步合入前需要过的关卡conversion-recipes.md 的五个配方产出的 SFT/DPO 行在真正进入训练前还要满足dataset-curation的 Phase 2 退出清单SKILL.md格式匹配目标方法上文的 Format Selection 表模板在拼接packing之前应用loss 只掩码到 assistant 轮解码并人工阅读 5–10 条 packed 序列最终混合中真实数据占比 ≥25%数据集卡六字段齐全——Provenance可回溯到trace-to-training-data输出、Counts、合成/真实比例、去重方法、所用模板、packing 配置。此外 Hygiene 节还要求任何行合入前需运行密钥/PII 扫描生产日志来源的轨迹可能携带凭据、API key、token 或客户数据敏感字段在脱敏后仍残留则 fail-closed 丢弃该行并针对已有训练集做去重精确匹配或 embedding 相似度匹配dataset-curation的 dedup method 字段——不只是对新转换行内部去重。preference-optimization的 references/method-configs.md 提供了 DPO 训练消费这些偏好对的具体配置模板DPOConfig使用beta0.1、learning_rate7e-75e-7–1e-6 区间低于产出该 checkpoint 的 SFT 学习率、1–2 epochref_modelNone表示 TRL 从model派生冻结参考模型迭代 on-policy DPO 的后续轮次则显式传入上一轮 checkpoint 作为ref_model。五个配方一览配方输入输出形态关键筛选原则1. Graded Trace → SFT单条通过阈值、带reward的轨迹messagesChatML保留 top-reward 分位非所有通过者2. 双轨迹 → DPO 对同一task_id的两条轨迹prompt/chosen/rejectedchosen最高分rejectedμ−2σ 最近者非最小值3. 纠错记录 → SFT失败轨迹 人类修正messages替换失败 assistant 轮人类已验证跳过奖励阈值4. 拒绝采样循环同一 prompt 的 n 条候选messages保留 topkeep_fractionn≤32按奖励分布调keep_fraction5. Goldens 留出校验待合入候选行 goldens.jsonl过滤后的行保留task_id期间执行golden ID 泄漏会静默抬高评测分数延伸阅读均位于本仓库plugins/llm-finetuning插件下本 skill 主文档skills/trace-to-training-data/SKILL.md——SFT From Traces、Preference Pairs From Traces、Hygiene 三节的完整原则阐述上游评分引擎skills/eval-harness-first/SKILL.md——runs/run-id/results.json与eval/goldens.jsonl的目录契约下游目标格式skills/dataset-curation/references/formats-and-templates.md——ChatML、DPO 对、KTO、GRPO 各格式的完整 JSONL 示例偏好优化消费端skills/preference-optimization/SKILL.md 及 references/method-configs.md——μ−2σ 公式归属地与 DPO/ORPO/KTO/SimPO 配置基础模型选择skills/finetuning-method-selection/references/model-catalog.md——本插件唯一允许指定具体模型名的文件【免费下载链接】agentsMulti-harness agentic plugin marketplace for Claude Code, Codex, Cursor, OpenCode, GitHub Copilot, and Google Antigravity项目地址: https://gitcode.com/GitHub_Trending/agents24/agents创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考