尧图网站建设 
MariaDB 集成指南在 Haystack 中构建基于向量检索与全文检索的 RAG 应用【免费下载链接】haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.项目地址: https://gitcode.com/GitHub_Trending/ha/haystackMariaDB 11.7 原生引入VECTOR数据类型与 MHNSW 索引使得数据库无需任何扩展即可直接执行高效的向量相似度检索。Haystack 的 MariaDB 集成mariadb-haystack在此基础上提供了MariaDBDocumentStore文档存储、MariaDBEmbeddingRetriever向量检索器与MariaDBKeywordRetriever关键词检索器覆盖写库—向量检索—全文检索—元数据过滤的完整链路。本文将以 API 参考文档 docs-website/reference_versioned_docs/version-2.23/integrations-api/mariadb.md 为核心骨架结合配套的组件文档与仓库内类型定义系统讲解环境搭建、三类组件的完整参数、串联进 RAG Pipeline 的实战写法以及背后的索引与距离函数原理。集成概览三个组件覆盖存—取—滤全流程MariaDB 集成在mariadb-haystack包中提供三个核心组件全部位于haystack_integrations命名空间下组件模块路径检索方式底层数据库能力MariaDBDocumentStorehaystack_integrations.document_stores.mariadb文档写入/过滤/计数/删除VECTOR数据类型 MATCH ... AGAINST全文索引MariaDBEmbeddingRetrieverhaystack_integrations.components.retrievers.mariadb向量相似度检索VEC_DISTANCE_COSINE/VEC_DISTANCE_EUCLIDEAN MHNSW 索引MariaDBKeywordRetrieverhaystack_integrations.components.retrievers.mariadb全文关键词检索MATCH ... AGAINST自然语言模式 content列的 FULLTEXT 索引配套的官方使用文档位于 docs-website/docs/document-stores/mariadbdocumentstore.mdx、docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdx 与 docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdx。需要说明的是mariadb-haystack集成本体托管在独立的haystack-core-integrations仓库本仓库haystack 主仓库中可直接确认的是集成所依赖的通用类型定义——例如DuplicatePolicy枚举定义于 haystack/document_stores/types/policy.pyFilterPolicy枚举与过滤合并逻辑定义于 haystack/document_stores/types/filter_policy.py元数据过滤的完整语法规范见 docs-website/docs/concepts/metadata-filtering.mdx。环境准备启动 MariaDB 11.7 并安装集成由于三个组件都依赖 MariaDB 11.7 的原生VECTOR能力第一步是准备一个满足版本要求的数据库实例。官方文档推荐用 Docker 快速启动docker run -d -p 3306:3306 \ -e MARIADB_ROOT_PASSWORDsecret \ -e MARIADB_DATABASEhaystack \ -e MARIADB_USERhaystack \ -e MARIADB_PASSWORDsecret \ mariadb:11.7这里同时创建了名为haystack的数据库和用户haystack密码均为secret宿主机 3306 端口映射到容器。随后安装 Python 集成包pip install mariadb-haystackmariadb驱动是一个需要从源码编译的 C 扩展因此系统必须先具备 MariaDB Connector/C 库提供mariadb_config# Ubuntu / Debian sudo apt-get install -y libmariadb-dev # macOS brew install mariadb-connector-c最后设置凭据环境变量——MariaDBDocumentStore的user与password参数默认从MARIADB_USER与MARIADB_PASSWORD读取export MARIADB_USERhaystack export MARIADB_PASSWORDsecretMariaDBDocumentStore表结构、连接参数与数据操作MariaDBDocumentStore是一个基于 MariaDB 11.7 原生VECTOR支持的文档存储。它使用VECTOR数据类型配合 MHNSW 索引做近似最近邻ANN向量检索同时用MATCH ... AGAINST完成全文关键词检索。构造函数与完整参数from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore store MariaDBDocumentStore( host127.0.0.1, port3306, databasehaystack, embedding_dimension768, )完整签名如下__init__( *, host: str 127.0.0.1, port: int 3306, database: str haystack, user: Secret Secret.from_env_var(MARIADB_USER), password: Secret Secret.from_env_var(MARIADB_PASSWORD), table_name: str haystack_documents, recreate_table: bool False, embedding_dimension: int 768, distance: str cosine, create_vector_index: bool False ) - None各参数的作用与限制host默认127.0.0.1MariaDB 主机地址。port默认3306MariaDB 端口。database默认haystack数据库名称。user/passwordSecret类型数据库凭据默认分别从环境变量MARIADB_USER与MARIADB_PASSWORD读取也支持Secret.from_token等方式显式传入。table_name默认haystack_documents存放文档的表名只能包含字母、数字与下划线。recreate_table默认False初始化时是否先删表再重建开启会删除表中全部数据务必谨慎。embedding_dimension默认768embedding 向量维度。仅在建表时生效对已存在的表无效。distance默认cosine向量距离函数取值cosine对应VEC_DISTANCE_COSINE或euclidean对应VEC_DISTANCE_EUCLIDEAN。同样仅在建表时生效。create_vector_index默认False为True时在建表阶段创建 MHNSW 向量索引以加速 ANN 检索。前提是每条文档都必须有非空 embedding否则写入会报错。依旧仅在建表时生效。官方文档特别强调见 docs-website/docs/document-stores/mariadbdocumentstore.mdxembedding_dimension、distance、create_vector_index这三个参数只在表首次创建或recreate_tableTrue重建时应用之后修改不会作用于已存在的表。因此生产环境中应在一开始就规划好向量维度、距离度量和是否启用向量索引需要调整时只能重建表。写入与基础操作import os from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack import Document os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret document_store MariaDBDocumentStore( port3306, databasehaystack, embedding_dimension768, distancecosine, ) document_store.write_documents( [ Document(contentThis is first, embedding[0.1] * 768), Document(contentThis is second, embedding[0.3] * 768), ], ) print(document_store.count_documents())write_documents的完整签名是write_documents( documents: list[Document], policy: DuplicatePolicy DuplicatePolicy.NONE ) - int其中policy使用 Haystack 主仓库中定义的DuplicatePolicy枚举haystack/document_stores/types/policy.py取值如下枚举值行为DuplicatePolicy.NONE默认值不显式声明重复策略DuplicatePolicy.SKIP遇到同 id 文档时跳过不覆盖DuplicatePolicy.OVERWRITE遇到同 id 文档时覆盖写入DuplicatePolicy.FAIL遇到同 id 文档时报DuplicateDocumentError文档存储还提供以下数据管理方法count_documents() - int返回当前存储中的文档数量。filter_documents(filters: dict[str, Any] | None None) - list[Document]按元数据过滤条件返回文档语法遵循 Haystack 的元数据过滤规范见 docs-website/docs/concepts/metadata-filtering.mdx。若filters不是字典会抛TypeError语法非法则抛ValueError。delete_documents(document_ids: list[str]) - None按文档 id 批量删除。delete_table() - None直接删除整个文档表。close() - None释放同步资源。to_dict()/from_dict(data)序列化与反序列化用于 Pipeline 的 YAML/JSON 持久化与加载。MariaDBEmbeddingRetriever基于向量相似度的检索MariaDBEmbeddingRetriever通过向量相似度检索从MariaDBDocumentStore取回文档底层调用 MariaDB 原生VEC_DISTANCE_COSINE或VEC_DISTANCE_EUCLIDEAN函数配合 MHNSW 索引实现高效的近似最近邻搜索。独立使用from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBEmbeddingRetriever store MariaDBDocumentStore(host127.0.0.1, databasehaystack, embedding_dimension768) retriever MariaDBEmbeddingRetriever(document_storestore, top_k5) result retriever.run(query_embedding[0.1] * 768) documents result[documents]构造参数与 run 参数__init__( *, document_store: MariaDBDocumentStore, filters: dict[str, Any] | None None, top_k: int 10, score_threshold: float | None None, filter_policy: str | FilterPolicy FilterPolicy.REPLACE ) - Nonedocument_store必须是一个MariaDBDocumentStore实例否则抛ValueError。filters默认 Haystack 元数据过滤条件作用于每一次查询用于收窄搜索空间。top_k默认10最多返回的文档数量。score_threshold分数下限低于该分数的文档被排除。filter_policy默认FilterPolicy.REPLACE定义运行时过滤条件与初始化过滤条件的交互方式。run方法的完整签名run( query_embedding: list[float], filters: dict[str, Any] | None None, top_k: int | None None, score_threshold: float | None None, ) - dict[str, list[Document]]query_embedding查询向量float 列表。filters运行时过滤条件按filter_policy与初始化过滤条件合并。top_k/score_threshold可覆盖构造时设定的值。返回值包含documents键的字典值为按相关性排序的Document列表。filter_policy 的语义filter_policy取自主仓库的FilterPolicy枚举haystack/document_stores/types/filter_policy.pyFilterPolicy.REPLACE默认运行时过滤条件直接替换初始化时的过滤条件。FilterPolicy.MERGE运行时过滤条件与初始化过滤条件合并同字段冲突时运行时值覆盖初始化值。apply_filter_policy是合并的核心实现当两种过滤条件都是比较型过滤含field/operator/value三键时通过combine_two_comparison_filters用逻辑运算符组合当涉及逻辑型过滤含operator/conditions时则根据双方 operator 是否一致决定合并条件还是降级为仅采用运行时过滤。这意味着 MERGE 模式下初始化过滤可以作为全局默认约束例如限定meta.tenant而每次查询再叠加动态条件。在 Pipeline 中与 Embedder 配合MariaDBEmbeddingRetriever典型位置是 Text Embedder 之后、PromptBuilder之前RAG 场景或作为语义搜索管线的末组件。嵌入的生成依赖 embedder 组件例如SentenceTransformersTextEmbedder/SentenceTransformersDocumentEmbedder对应集成包sentence-transformers-haystack。一个完整的索引 查询双管线示例出自 docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdximport os from haystack import Document, Pipeline from haystack_integrations.components.embedders.sentence_transformers import ( SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder, ) from haystack.document_stores.types import DuplicatePolicy from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import ( MariaDBEmbeddingRetriever, ) os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret document_store MariaDBDocumentStore( embedding_dimension768, distancecosine, ) documents [ Document(contentThere are over 7,000 languages spoken around the world today.), Document(contentElephants have been observed to recognize themselves in mirrors.), Document(contentBioluminescent waves can be seen in the Maldives and Puerto Rico.), ] document_embedder SentenceTransformersDocumentEmbedder() documents_with_embeddings document_embedder.run(documents) document_store.write_documents( documents_with_embeddings.get(documents), policyDuplicatePolicy.OVERWRITE, ) query_pipeline Pipeline() query_pipeline.add_component(text_embedder, SentenceTransformersTextEmbedder()) query_pipeline.add_component( retriever, MariaDBEmbeddingRetriever(document_storedocument_store), ) query_pipeline.connect(text_embedder.embedding, retriever.query_embedding) result query_pipeline.run( {text_embedder: {text: How many languages are there?}} ) print(result[retriever][documents][0])需要注意embedding_dimension必须与 embedder 输出的向量维度严格一致示例中 Sentence Transformers 默认 384 维而文档示例用了 768 维需按实际模型调整若启用create_vector_indexTrue写入的每条文档都必须带非空 embedding。MariaDBKeywordRetriever基于全文索引的关键词检索MariaDBKeywordRetriever使用 MariaDB 内置全文检索能力通过MATCH ... AGAINST自然语言模式在content列的 FULLTEXT 索引上匹配查询词无需向量化即可工作。独立使用from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever store MariaDBDocumentStore(host127.0.0.1, databasehaystack, embedding_dimension768) retriever MariaDBKeywordRetriever(document_storestore, top_k5) result retriever.run(queryclimate change) documents result[documents]构造参数与 run 参数__init__( *, document_store: MariaDBDocumentStore, filters: dict[str, Any] | None None, top_k: int 10, filter_policy: str | FilterPolicy FilterPolicy.REPLACE ) - None参数语义与 embedding 检索器一致document_store必须是MariaDBDocumentStore否则抛ValueErrorfilters为默认元数据过滤top_k默认 10filter_policy默认REPLACE。区别在于 keyword 检索器没有score_threshold参数。run方法的完整签名run( query: str, filters: dict[str, Any] | None None, top_k: int | None None ) - dict[str, list[Document]]query关键词查询字符串。filters运行时过滤条件按filter_policy合并。top_k可覆盖构造时的取值。返回值包含documents键的字典值为按相关性排序的Document列表。在 RAG Pipeline 中的完整用法关键词检索非常适合无需向量化的轻量 RAG。以下完整示例出自 docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdx演示了MariaDBKeywordRetriever与ChatPromptBuilder、OpenAIChatGenerator、AnswerBuilder串联的检索增强生成流程import os from haystack import Document, Pipeline from haystack.components.builders import AnswerBuilder, ChatPromptBuilder from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack.document_stores.types import DuplicatePolicy from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret os.environ[OPENAI_API_KEY] your-openai-api-key prompt_template [ ChatMessage.from_user( Given these documents, answer the question. Documents: {% for doc in documents %} {{ doc.content }} {% endfor %} Question: {{question}} Answer: ), ] document_store MariaDBDocumentStore() documents [ Document(contentThere are over 7,000 languages spoken around the world today.), Document(contentElephants have been observed to recognize themselves in mirrors.), Document(contentBioluminescent waves can be seen in the Maldives and Puerto Rico.), ] document_store.write_documents(documentsdocuments, policyDuplicatePolicy.SKIP) retriever MariaDBKeywordRetriever(document_storedocument_store) rag_pipeline Pipeline() rag_pipeline.add_component(nameretriever, instanceretriever) rag_pipeline.add_component( instanceChatPromptBuilder(templateprompt_template, required_variables*), nameprompt_builder, ) rag_pipeline.add_component(instanceOpenAIChatGenerator(), namellm) rag_pipeline.add_component(instanceAnswerBuilder(), nameanswer_builder) rag_pipeline.connect(retriever, prompt_builder.documents) rag_pipeline.connect(prompt_builder.prompt, llm.messages) rag_pipeline.connect(llm.replies, answer_builder.replies) rag_pipeline.connect(retriever, answer_builder.documents) question languages spoken around the world today result rag_pipeline.run( { retriever: {query: question}, prompt_builder: {question: question}, answer_builder: {query: question}, } ) print(result[answer_builder])该示例中写入文档时使用了DuplicatePolicy.SKIP重复 id 跳过检索器输出documents同时接入prompt_builder和answer_builder由 LLM 依据检索到的上下文生成最终答案。序列化与管线持久化三个组件都实现了to_dict()与from_dict()MariaDBDocumentStore.to_dict() - dict[str, Any]将存储配置序列化为字典from_dict(data)还原实例。MariaDBEmbeddingRetriever.to_dict()/from_dict(data)序列化 / 反序列化检索器。MariaDBKeywordRetriever.to_dict()/from_dict(data)同上。这使 MariaDB 组件可以无缝融入 Haystack 的 YAML/JSON 管线定义与Pipeline.loads()/Pipeline.dumps()持久化机制。注意序列化不会导出表内数据只包含连接与组件配置。序列化最佳实践基于以上原理与文档约束使用 MariaDB 集成时建议遵循以下要点版本与索引规划前置embedding_dimension、distance、create_vector_index只在建表时生效需在初始化前确定。需要 MHNSW 加速 ANN 检索时初始化MariaDBDocumentStore即传入create_vector_indexTrue并保证后续所有写入文档均含非空 embedding。向量维度对齐embedding_dimension必须与所选 embedder如 Sentence Transformers、OpenAI Embedder 等实际输出维度一致否则相似度计算会出错。凭据走环境变量默认从MARIADB_USER/MARIADB_PASSWORD读取避免把密码硬编码进代码或序列化文件。区分两种检索能力有 embedding 数据的场景用MariaDBEmbeddingRetriever做语义检索文本场景如传统关键词匹配、无向量化成本诉求用MariaDBKeywordRetriever两者都支持filters与filter_policy动态收窄范围。重复数据处理根据业务选择DuplicatePolicy——索引任务常用OVERWRITE或SKIP需要严格校验时用FAIL。元数据过滤filters遵循 Haystack 统一过滤语法比较型field/operator/value与逻辑型operator/conditions均可使用具体语法见 docs-website/docs/concepts/metadata-filtering.mdx。参考资料API 参考本文核心来源docs-website/reference_versioned_docs/version-2.23/integrations-api/mariadb.md文档存储使用指南docs-website/docs/document-stores/mariadbdocumentstore.mdx向量检索器指南docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdx关键词检索器指南docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdxDuplicatePolicy定义haystack/document_stores/types/policy.pyFilterPolicy与过滤合并实现haystack/document_stores/types/filter_policy.py元数据过滤语法docs-website/docs/concepts/metadata-filtering.mdx【免费下载链接】haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.项目地址: https://gitcode.com/GitHub_Trending/ha/haystack创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 MariaDB 集成指南：在 Haystack 中构建基于向量检索与全文检索的 RAG 应用
MariaDB 集成指南在 Haystack 中构建基于向量检索与全文检索的 RAG 应用【免费下载链接】haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.项目地址: https://gitcode.com/GitHub_Trending/ha/haystackMariaDB 11.7 原生引入VECTOR数据类型与 MHNSW 索引使得数据库无需任何扩展即可直接执行高效的向量相似度检索。Haystack 的 MariaDB 集成mariadb-haystack在此基础上提供了MariaDBDocumentStore文档存储、MariaDBEmbeddingRetriever向量检索器与MariaDBKeywordRetriever关键词检索器覆盖写库—向量检索—全文检索—元数据过滤的完整链路。本文将以 API 参考文档 docs-website/reference_versioned_docs/version-2.23/integrations-api/mariadb.md 为核心骨架结合配套的组件文档与仓库内类型定义系统讲解环境搭建、三类组件的完整参数、串联进 RAG Pipeline 的实战写法以及背后的索引与距离函数原理。集成概览三个组件覆盖存—取—滤全流程MariaDB 集成在mariadb-haystack包中提供三个核心组件全部位于haystack_integrations命名空间下组件模块路径检索方式底层数据库能力MariaDBDocumentStorehaystack_integrations.document_stores.mariadb文档写入/过滤/计数/删除VECTOR数据类型 MATCH ... AGAINST全文索引MariaDBEmbeddingRetrieverhaystack_integrations.components.retrievers.mariadb向量相似度检索VEC_DISTANCE_COSINE/VEC_DISTANCE_EUCLIDEAN MHNSW 索引MariaDBKeywordRetrieverhaystack_integrations.components.retrievers.mariadb全文关键词检索MATCH ... AGAINST自然语言模式 content列的 FULLTEXT 索引配套的官方使用文档位于 docs-website/docs/document-stores/mariadbdocumentstore.mdx、docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdx 与 docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdx。需要说明的是mariadb-haystack集成本体托管在独立的haystack-core-integrations仓库本仓库haystack 主仓库中可直接确认的是集成所依赖的通用类型定义——例如DuplicatePolicy枚举定义于 haystack/document_stores/types/policy.pyFilterPolicy枚举与过滤合并逻辑定义于 haystack/document_stores/types/filter_policy.py元数据过滤的完整语法规范见 docs-website/docs/concepts/metadata-filtering.mdx。环境准备启动 MariaDB 11.7 并安装集成由于三个组件都依赖 MariaDB 11.7 的原生VECTOR能力第一步是准备一个满足版本要求的数据库实例。官方文档推荐用 Docker 快速启动docker run -d -p 3306:3306 \ -e MARIADB_ROOT_PASSWORDsecret \ -e MARIADB_DATABASEhaystack \ -e MARIADB_USERhaystack \ -e MARIADB_PASSWORDsecret \ mariadb:11.7这里同时创建了名为haystack的数据库和用户haystack密码均为secret宿主机 3306 端口映射到容器。随后安装 Python 集成包pip install mariadb-haystackmariadb驱动是一个需要从源码编译的 C 扩展因此系统必须先具备 MariaDB Connector/C 库提供mariadb_config# Ubuntu / Debian sudo apt-get install -y libmariadb-dev # macOS brew install mariadb-connector-c最后设置凭据环境变量——MariaDBDocumentStore的user与password参数默认从MARIADB_USER与MARIADB_PASSWORD读取export MARIADB_USERhaystack export MARIADB_PASSWORDsecretMariaDBDocumentStore表结构、连接参数与数据操作MariaDBDocumentStore是一个基于 MariaDB 11.7 原生VECTOR支持的文档存储。它使用VECTOR数据类型配合 MHNSW 索引做近似最近邻ANN向量检索同时用MATCH ... AGAINST完成全文关键词检索。构造函数与完整参数from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore store MariaDBDocumentStore( host127.0.0.1, port3306, databasehaystack, embedding_dimension768, )完整签名如下__init__( *, host: str 127.0.0.1, port: int 3306, database: str haystack, user: Secret Secret.from_env_var(MARIADB_USER), password: Secret Secret.from_env_var(MARIADB_PASSWORD), table_name: str haystack_documents, recreate_table: bool False, embedding_dimension: int 768, distance: str cosine, create_vector_index: bool False ) - None各参数的作用与限制host默认127.0.0.1MariaDB 主机地址。port默认3306MariaDB 端口。database默认haystack数据库名称。user/passwordSecret类型数据库凭据默认分别从环境变量MARIADB_USER与MARIADB_PASSWORD读取也支持Secret.from_token等方式显式传入。table_name默认haystack_documents存放文档的表名只能包含字母、数字与下划线。recreate_table默认False初始化时是否先删表再重建开启会删除表中全部数据务必谨慎。embedding_dimension默认768embedding 向量维度。仅在建表时生效对已存在的表无效。distance默认cosine向量距离函数取值cosine对应VEC_DISTANCE_COSINE或euclidean对应VEC_DISTANCE_EUCLIDEAN。同样仅在建表时生效。create_vector_index默认False为True时在建表阶段创建 MHNSW 向量索引以加速 ANN 检索。前提是每条文档都必须有非空 embedding否则写入会报错。依旧仅在建表时生效。官方文档特别强调见 docs-website/docs/document-stores/mariadbdocumentstore.mdxembedding_dimension、distance、create_vector_index这三个参数只在表首次创建或recreate_tableTrue重建时应用之后修改不会作用于已存在的表。因此生产环境中应在一开始就规划好向量维度、距离度量和是否启用向量索引需要调整时只能重建表。写入与基础操作import os from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack import Document os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret document_store MariaDBDocumentStore( port3306, databasehaystack, embedding_dimension768, distancecosine, ) document_store.write_documents( [ Document(contentThis is first, embedding[0.1] * 768), Document(contentThis is second, embedding[0.3] * 768), ], ) print(document_store.count_documents())write_documents的完整签名是write_documents( documents: list[Document], policy: DuplicatePolicy DuplicatePolicy.NONE ) - int其中policy使用 Haystack 主仓库中定义的DuplicatePolicy枚举haystack/document_stores/types/policy.py取值如下枚举值行为DuplicatePolicy.NONE默认值不显式声明重复策略DuplicatePolicy.SKIP遇到同 id 文档时跳过不覆盖DuplicatePolicy.OVERWRITE遇到同 id 文档时覆盖写入DuplicatePolicy.FAIL遇到同 id 文档时报DuplicateDocumentError文档存储还提供以下数据管理方法count_documents() - int返回当前存储中的文档数量。filter_documents(filters: dict[str, Any] | None None) - list[Document]按元数据过滤条件返回文档语法遵循 Haystack 的元数据过滤规范见 docs-website/docs/concepts/metadata-filtering.mdx。若filters不是字典会抛TypeError语法非法则抛ValueError。delete_documents(document_ids: list[str]) - None按文档 id 批量删除。delete_table() - None直接删除整个文档表。close() - None释放同步资源。to_dict()/from_dict(data)序列化与反序列化用于 Pipeline 的 YAML/JSON 持久化与加载。MariaDBEmbeddingRetriever基于向量相似度的检索MariaDBEmbeddingRetriever通过向量相似度检索从MariaDBDocumentStore取回文档底层调用 MariaDB 原生VEC_DISTANCE_COSINE或VEC_DISTANCE_EUCLIDEAN函数配合 MHNSW 索引实现高效的近似最近邻搜索。独立使用from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBEmbeddingRetriever store MariaDBDocumentStore(host127.0.0.1, databasehaystack, embedding_dimension768) retriever MariaDBEmbeddingRetriever(document_storestore, top_k5) result retriever.run(query_embedding[0.1] * 768) documents result[documents]构造参数与 run 参数__init__( *, document_store: MariaDBDocumentStore, filters: dict[str, Any] | None None, top_k: int 10, score_threshold: float | None None, filter_policy: str | FilterPolicy FilterPolicy.REPLACE ) - Nonedocument_store必须是一个MariaDBDocumentStore实例否则抛ValueError。filters默认 Haystack 元数据过滤条件作用于每一次查询用于收窄搜索空间。top_k默认10最多返回的文档数量。score_threshold分数下限低于该分数的文档被排除。filter_policy默认FilterPolicy.REPLACE定义运行时过滤条件与初始化过滤条件的交互方式。run方法的完整签名run( query_embedding: list[float], filters: dict[str, Any] | None None, top_k: int | None None, score_threshold: float | None None, ) - dict[str, list[Document]]query_embedding查询向量float 列表。filters运行时过滤条件按filter_policy与初始化过滤条件合并。top_k/score_threshold可覆盖构造时设定的值。返回值包含documents键的字典值为按相关性排序的Document列表。filter_policy 的语义filter_policy取自主仓库的FilterPolicy枚举haystack/document_stores/types/filter_policy.pyFilterPolicy.REPLACE默认运行时过滤条件直接替换初始化时的过滤条件。FilterPolicy.MERGE运行时过滤条件与初始化过滤条件合并同字段冲突时运行时值覆盖初始化值。apply_filter_policy是合并的核心实现当两种过滤条件都是比较型过滤含field/operator/value三键时通过combine_two_comparison_filters用逻辑运算符组合当涉及逻辑型过滤含operator/conditions时则根据双方 operator 是否一致决定合并条件还是降级为仅采用运行时过滤。这意味着 MERGE 模式下初始化过滤可以作为全局默认约束例如限定meta.tenant而每次查询再叠加动态条件。在 Pipeline 中与 Embedder 配合MariaDBEmbeddingRetriever典型位置是 Text Embedder 之后、PromptBuilder之前RAG 场景或作为语义搜索管线的末组件。嵌入的生成依赖 embedder 组件例如SentenceTransformersTextEmbedder/SentenceTransformersDocumentEmbedder对应集成包sentence-transformers-haystack。一个完整的索引 查询双管线示例出自 docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdximport os from haystack import Document, Pipeline from haystack_integrations.components.embedders.sentence_transformers import ( SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder, ) from haystack.document_stores.types import DuplicatePolicy from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import ( MariaDBEmbeddingRetriever, ) os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret document_store MariaDBDocumentStore( embedding_dimension768, distancecosine, ) documents [ Document(contentThere are over 7,000 languages spoken around the world today.), Document(contentElephants have been observed to recognize themselves in mirrors.), Document(contentBioluminescent waves can be seen in the Maldives and Puerto Rico.), ] document_embedder SentenceTransformersDocumentEmbedder() documents_with_embeddings document_embedder.run(documents) document_store.write_documents( documents_with_embeddings.get(documents), policyDuplicatePolicy.OVERWRITE, ) query_pipeline Pipeline() query_pipeline.add_component(text_embedder, SentenceTransformersTextEmbedder()) query_pipeline.add_component( retriever, MariaDBEmbeddingRetriever(document_storedocument_store), ) query_pipeline.connect(text_embedder.embedding, retriever.query_embedding) result query_pipeline.run( {text_embedder: {text: How many languages are there?}} ) print(result[retriever][documents][0])需要注意embedding_dimension必须与 embedder 输出的向量维度严格一致示例中 Sentence Transformers 默认 384 维而文档示例用了 768 维需按实际模型调整若启用create_vector_indexTrue写入的每条文档都必须带非空 embedding。MariaDBKeywordRetriever基于全文索引的关键词检索MariaDBKeywordRetriever使用 MariaDB 内置全文检索能力通过MATCH ... AGAINST自然语言模式在content列的 FULLTEXT 索引上匹配查询词无需向量化即可工作。独立使用from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever store MariaDBDocumentStore(host127.0.0.1, databasehaystack, embedding_dimension768) retriever MariaDBKeywordRetriever(document_storestore, top_k5) result retriever.run(queryclimate change) documents result[documents]构造参数与 run 参数__init__( *, document_store: MariaDBDocumentStore, filters: dict[str, Any] | None None, top_k: int 10, filter_policy: str | FilterPolicy FilterPolicy.REPLACE ) - None参数语义与 embedding 检索器一致document_store必须是MariaDBDocumentStore否则抛ValueErrorfilters为默认元数据过滤top_k默认 10filter_policy默认REPLACE。区别在于 keyword 检索器没有score_threshold参数。run方法的完整签名run( query: str, filters: dict[str, Any] | None None, top_k: int | None None ) - dict[str, list[Document]]query关键词查询字符串。filters运行时过滤条件按filter_policy合并。top_k可覆盖构造时的取值。返回值包含documents键的字典值为按相关性排序的Document列表。在 RAG Pipeline 中的完整用法关键词检索非常适合无需向量化的轻量 RAG。以下完整示例出自 docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdx演示了MariaDBKeywordRetriever与ChatPromptBuilder、OpenAIChatGenerator、AnswerBuilder串联的检索增强生成流程import os from haystack import Document, Pipeline from haystack.components.builders import AnswerBuilder, ChatPromptBuilder from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack.document_stores.types import DuplicatePolicy from haystack_integrations.document_stores.mariadb import MariaDBDocumentStore from haystack_integrations.components.retrievers.mariadb import MariaDBKeywordRetriever os.environ[MARIADB_USER] haystack os.environ[MARIADB_PASSWORD] secret os.environ[OPENAI_API_KEY] your-openai-api-key prompt_template [ ChatMessage.from_user( Given these documents, answer the question. Documents: {% for doc in documents %} {{ doc.content }} {% endfor %} Question: {{question}} Answer: ), ] document_store MariaDBDocumentStore() documents [ Document(contentThere are over 7,000 languages spoken around the world today.), Document(contentElephants have been observed to recognize themselves in mirrors.), Document(contentBioluminescent waves can be seen in the Maldives and Puerto Rico.), ] document_store.write_documents(documentsdocuments, policyDuplicatePolicy.SKIP) retriever MariaDBKeywordRetriever(document_storedocument_store) rag_pipeline Pipeline() rag_pipeline.add_component(nameretriever, instanceretriever) rag_pipeline.add_component( instanceChatPromptBuilder(templateprompt_template, required_variables*), nameprompt_builder, ) rag_pipeline.add_component(instanceOpenAIChatGenerator(), namellm) rag_pipeline.add_component(instanceAnswerBuilder(), nameanswer_builder) rag_pipeline.connect(retriever, prompt_builder.documents) rag_pipeline.connect(prompt_builder.prompt, llm.messages) rag_pipeline.connect(llm.replies, answer_builder.replies) rag_pipeline.connect(retriever, answer_builder.documents) question languages spoken around the world today result rag_pipeline.run( { retriever: {query: question}, prompt_builder: {question: question}, answer_builder: {query: question}, } ) print(result[answer_builder])该示例中写入文档时使用了DuplicatePolicy.SKIP重复 id 跳过检索器输出documents同时接入prompt_builder和answer_builder由 LLM 依据检索到的上下文生成最终答案。序列化与管线持久化三个组件都实现了to_dict()与from_dict()MariaDBDocumentStore.to_dict() - dict[str, Any]将存储配置序列化为字典from_dict(data)还原实例。MariaDBEmbeddingRetriever.to_dict()/from_dict(data)序列化 / 反序列化检索器。MariaDBKeywordRetriever.to_dict()/from_dict(data)同上。这使 MariaDB 组件可以无缝融入 Haystack 的 YAML/JSON 管线定义与Pipeline.loads()/Pipeline.dumps()持久化机制。注意序列化不会导出表内数据只包含连接与组件配置。序列化最佳实践基于以上原理与文档约束使用 MariaDB 集成时建议遵循以下要点版本与索引规划前置embedding_dimension、distance、create_vector_index只在建表时生效需在初始化前确定。需要 MHNSW 加速 ANN 检索时初始化MariaDBDocumentStore即传入create_vector_indexTrue并保证后续所有写入文档均含非空 embedding。向量维度对齐embedding_dimension必须与所选 embedder如 Sentence Transformers、OpenAI Embedder 等实际输出维度一致否则相似度计算会出错。凭据走环境变量默认从MARIADB_USER/MARIADB_PASSWORD读取避免把密码硬编码进代码或序列化文件。区分两种检索能力有 embedding 数据的场景用MariaDBEmbeddingRetriever做语义检索文本场景如传统关键词匹配、无向量化成本诉求用MariaDBKeywordRetriever两者都支持filters与filter_policy动态收窄范围。重复数据处理根据业务选择DuplicatePolicy——索引任务常用OVERWRITE或SKIP需要严格校验时用FAIL。元数据过滤filters遵循 Haystack 统一过滤语法比较型field/operator/value与逻辑型operator/conditions均可使用具体语法见 docs-website/docs/concepts/metadata-filtering.mdx。参考资料API 参考本文核心来源docs-website/reference_versioned_docs/version-2.23/integrations-api/mariadb.md文档存储使用指南docs-website/docs/document-stores/mariadbdocumentstore.mdx向量检索器指南docs-website/docs/pipeline-components/retrievers/mariadbembeddingretriever.mdx关键词检索器指南docs-website/docs/pipeline-components/retrievers/mariadbkeywordretriever.mdxDuplicatePolicy定义haystack/document_stores/types/policy.pyFilterPolicy与过滤合并实现haystack/document_stores/types/filter_policy.py元数据过滤语法docs-website/docs/concepts/metadata-filtering.mdx【免费下载链接】haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.项目地址: https://gitcode.com/GitHub_Trending/ha/haystack创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考