Huly Virtual Network 核心概念指南深入理解 Hub-and-Spoke 虚拟网络的 Network、Agent、Container 与客户端通信机制【免费下载链接】platformHuly — All-in-One Project Management Platform (alternative to Linear, Jira, Slack, Notion, Motion)项目地址: https://gitcode.com/GitHub_Trending/platform80/platformHuly Virtual Network 是 HulyAll-in-One Project Management Platform中用于构建分布式系统的底层虚拟网络基础设施本文以其官方文档 CORE_CONCEPTS.md 为核心骨架结合仓库内hcengineering/network-core、network-client、network-server、network-backrpc等包的源码实现系统讲解其中心辐射hub-and-spoke架构、三大核心角色Network / Agent / Container、客户端通信模式、生命周期管理与端点寻址机制。读完本文你将掌握 Huly 虚拟网络的核心抽象模型、如何编写自定义容器与代理以及如何利用引用计数、健康检查与无状态容器实现高可用的分布式服务。架构总览中心辐射Hub-and-Spoke模型Huly Network 采用hub-and-spoke 架构包含三个主要组件Network中心枢纽网络服务器Network Server、路由器Router与容器注册表Container Registry的集合体是整个系统的协调中枢。Agents辐条承载并管理容器的工作进程可分布部署在多台机器上。Clients消费方请求容器、与容器通信的应用或服务。关键设计原则集中协调Centralized Coordination网络服务器统一协调所有 Agent 与容器。分布式执行Distributed Execution容器运行在 Agent 上Agent 可跨多台机器分布。动态发现Dynamic Discovery客户端通过网络发现并连接到容器无需外部服务注册中心Consul、etcd、ZooKeeper。自动生命周期Automatic Lifecycle网络基于客户端引用自动管理容器生命周期。故障容忍Fault Tolerance自动清理失效的 Agent 与孤儿容器orphaned containers。部署注意根据 foundations/net/README.md 的说明网络服务器本身是单实例、无 HA、单点故障的——只有 Agent 与容器支持高可用通过无状态容器注册与自动故障转移。生产环境中应使用进程守护systemd、PM2、Kubernetes restart 策略与快速重启机制Agent 与容器在网络服务重启后会自动重连。Network集中协调者Network是系统的中央协调者负责维护所有 Agent 及其能力的注册表追踪所有活跃容器及其所在位置将客户端请求路由到合适的容器管理容器生命周期创建、引用计数、清理提供服务发现与负载均衡向客户端广播系统变化事件。Network 接口定义文档中给出的核心接口与仓库源码 api/network.ts 中的Network接口基本一致interface Network { // Agent management register(record: AgentRecord, agent: NetworkAgent): PromiseContainerUuid[] unregister(agentId: AgentUuid): Promisevoid ping(agentId: AgentUuid): Promisevoid // Container management get(client: ClientUuid, kind: ContainerKind, options: GetOptions): Promise[ContainerUuid, ContainerEndpointRef] release(client: ClientUuid, uuid: ContainerUuid): Promisevoid list(kind?: ContainerKind): PromiseContainerRecord[] // Communication request(target: ContainerUuid, operation: string, data?: any): Promiseany // Discovery agents(): AgentRecord[] kinds(): ContainerKind[] }补充说明源码细节接口中ping的语义是标记一个 Agent/Client 为存活更新 lastSeen 时间戳且实际返回值为voidAgent 的健康检测由网络在每个 tick 周期调用checkAlive()完成见 network.ts。register在每次网络重启后都会被重新调用即Agent 应重连并重新注册源码注释明确说明。除上述方法外Network还实现了NetworkWithClients扩展接口addClient/removeClient/mapAgent/unmapAgent用于将客户端与 Agent、容器事件建立映射见 api/network.ts。网络服务器Network Server网络服务器监听 TCP 端口默认 3737并接受来自客户端与 Agent 的连接使用ZeroMQ进行高性能消息传递高效处理并发请求通过 ping/pong 维持连接健康。启动网络服务器示例来自文档与 examples/04-complete-production-setup.ts 中的生产用法一致import { NetworkImpl, TickManagerImpl } from hcengineering/network-core import { NetworkServer } from hcengineering/network-server const tickManager new TickManagerImpl(1000) // 1000 ticks/sec tickManager.start() const network new NetworkImpl(tickManager) const server new NetworkServer( network, tickManager, *, // Bind to all interfaces 3737 // Port ) console.log(Network server running on port 3737)TickManagerImpl是整个系统的心跳时钟NetworkImpl在构造时会向 tickManager 注册周期性任务间隔为aliveTimeout每次 tick 依次执行checkAlive()探测死亡 Agent与sendEvents()向客户端派发事件队列见 network.ts。Agents承载容器的工作节点Agents是工作进程负责承载并管理容器向网络注册以通告自身能力按需创建容器或预置无状态stateless容器处理容器生命周期启动、停止、健康检查在客户端与容器之间路由请求。Agent 能力声明每个 Agent 声明其可创建的容器种类container kinds。文档示例直接使用AgentImpl以教学为目的生产代码应使用客户端提供的serveAgent()详见下文// Note: For production code, use serveAgent() on the client // This example uses AgentImpl directly for educational purposes const agent new AgentImpl(my-agent as AgentUuid, { session: async (options) { /* create session container */ }, workspace: async (options) { /* create workspace container */ }, query: async (options) { /* create query container */ } })对应源码层面NetworkAgent接口api/agent.ts要求 Agent 提供稳定的唯一标识uuidAgent 重启后保持不变、Agent 连接端点endpoint、支持的种类集合kinds以及get/getContainer/list/request/terminate等方法。AgentRecord则用于注册通告包含agentId、endpoint、containers与kinds。Agent 注册Agent 通过NetworkAgentServer对外提供容器连接端口示例中为 3738再通过客户端register向网络通告自身import { NetworkAgentServer } from hcengineering/network-client const agentServer new NetworkAgentServer( tickManager, localhost, // Network host *, // Bind address 3738 // Agent port for container connections ) await agentServer.start(agent) // Register with network await client.register(agent)Agent 健康机制Agent 必须定期向网络发送 ping默认每 1 秒即pingInterval响应健康检查优雅关闭终止时清理其容器。若 Agent 在aliveTimeout默认 3 秒内未 ping网络将将该 Agent 标记为死亡移除其全部容器广播移除事件允许备用 Agent 接管针对无状态容器。以上默认值均可在源码 api/timeouts.ts 中直接确认export const timeouts { aliveTimeout: 3, // seconds - timeout for detecting dead agents/clients unusedContainerTimeout: 5, // seconds for container to be terminated because of being unused pingInterval: 1 // seconds - how often to ping agents }生产推荐serveAgent()仓库实际推荐的生产接入方式是客户端上的serveAgent()它把启动 Agent 端口服务 注册到网络封装为一步。例如 README.md 中的用法const client createNetworkClient(localhost:3737) await client.waitConnection(5000) await client.serveAgent(localhost:3738, { my-service: async (options, agentEndpoint) { const uuid options.uuid ?? ((container- Date.now()) as ContainerUuid) const container new MyServiceContainer(uuid) return { uuid, container, endpoint: containerOnAgentEndpointRef(agentEndpoint!, uuid) } } })containerOnAgentEndpointRef会将 Agent 端点转换为该容器对应的 routed 端点具体实现见 endpoints.ts。Containers系统的工作主力Containers是系统的工作主力它们实现具体业务逻辑处理来自客户端的请求维护内部状态向已连接客户端广播事件按需自动创建与销毁。Container 接口每个容器必须实现Container接口。文档版本与源码 containers.ts 中定义一致interface Container { // Handle incoming requests request(operation: string, data?: any, clientId?: ClientUuid): Promiseany // Health check ping(): Promisevoid // Cleanup and shutdown terminate(): Promisevoid // Event broadcasting support connect(clientId: ClientUuid, broadcast: (data: any) Promisevoid): void disconnect(clientId: ClientUuid): void // Optional termination callback onTerminated?(): void }源码中还定义了配套的ContainerFactory类型export type ContainerFactory ( request: GetOptions, agentEndpoint?: AgentEndpointRef ) Promise{ uuid: ContainerUuid, container: Container, endpoint: ContainerEndpointRef }即工厂方法接收请求选项与 Agent 端点返回容器实例、其 UUID 与端点引用。容器生命周期关键生命周期事件创建Creation客户端请求容器时调用工厂方法活跃Active容器运行并处理请求被引用Referenced至少一个客户端持有引用空闲超时Idle Timeout最后一个引用释放后容器继续存活containerTimeout源码中对应unusedContainerTimeout默认 5 秒终止Termination调用terminate()然后从网络注册表中移除。容器类型有状态容器动态创建按需创建常用于用户会话等场景。文档示例// Note: For production code, use serveAgent() on the client // This example uses AgentImpl directly for educational purposes const agent new AgentImpl(agent-1 as any, { user-session: async (options: GetOptions) { const uuid options.uuid ?? generateUuid() const container new UserSessionContainer(uuid, options) return { uuid, container, endpoint: session://agent1/${uuid} as any } } })无状态容器预置High Availability预创建以获得高可用const leaderContainer new LeaderContainer(leader-001 as ContainerUuid) agent.addStatelessContainer( leader-001 as ContainerUuid, leader as ContainerKind, leader://agent1/leader-001 as ContainerEndpointRef, leaderContainer )多个 Agent 可以注册相同的无状态容器 UUID网络接受第一个注册、拒绝其余注册从而实现自动故障转移failover。这一模式是 Huly Network 实现无需外部协调服务的领导者选举/高可用的关键仓库中 ha-stateless.spec.ts 与 HA_STATELESS_CONTAINERS.md 提供了对应验证与深入讲解。容器实现示例DataStoreContainer文档给出的完整示例展示了一个带事件广播能力的内存数据容器import type { Container, ContainerUuid, ClientUuid } from hcengineering/network-core class DataStoreContainer implements Container { private data new Mapstring, any() private connections new MapClientUuid, (data: any) Promisevoid() constructor(readonly uuid: ContainerUuid) {} async request(operation: string, data?: any): Promiseany { switch (operation) { case set: this.data.set(data.key, data.value) await this.broadcast({ type: dataChanged, key: data.key }) return { success: true } case get: return { value: this.data.get(data.key) } default: return { error: Unknown operation } } } async ping(): Promisevoid {} async terminate(): Promisevoid { this.data.clear() this.connections.clear() } connect(clientId: ClientUuid, broadcast: (data: any) Promisevoid): void { this.connections.set(clientId, broadcast) } disconnect(clientId: ClientUuid): void { this.connections.delete(clientId) } private async broadcast(event: any): Promisevoid { const promises Array.from(this.connections.values()).map((fn) fn(event)) await Promise.all(promises) } }需要更大规模、带监控与优雅关闭的生产级容器实现可参考 examples/04-complete-production-setup.ts 中的ProductionServiceContainer实现了process/healthCheck/getMetrics/simulateError等操作并记录错误率与连接数。Clients请求容器的一方Clients是应用或服务负责连接网络服务器按 kind 与可选条件请求容器向容器发送请求接收容器事件管理容器引用获取/释放。客户端连接import { createNetworkClient } from hcengineering/network-client const client createNetworkClient( localhost:3737, // Network address 3600 // Alive timeout in seconds (optional) ) await client.waitConnection(5000) // Wait up to 5 seconds补充源码 api/client.ts客户端在实例化时生成标识并无限期尝试连接waitConnection(timeout)用于确保在期限内完成首次连接timeout 0表示无限等待。创建时的第二个参数为客户端级别的 alive timeout秒可用于开发环境如 3600 秒 1 小时便于调试与生产环境默认 3 秒的差异化配置见 examples/custom-timeout-example.ts。请求容器客户端按kind和可选条件criteria请求容器// Get any container of this kind const ref await client.get(user-session as ContainerKind, {}) // Get specific container by UUID const ref await client.get(user-session as ContainerKind, { uuid: session-123 as ContainerUuid }) // Get container with labels const ref await client.get(workspace as ContainerKind, { labels: [premium, us-west] }) // Get container with extra data const ref await client.get(query-engine as ContainerKind, { extra: { database: analytics, userId: user-456 } })客户端 APIinterface NetworkClient { // Container management get(kind: ContainerKind, request: GetOptions): PromiseContainerReference list(kind?: ContainerKind): PromiseContainerRecord[] // Agent management register(agent: NetworkAgent): Promisevoid unregister(agentId: AgentUuid): Promisevoid // Discovery agents(): AgentRecord[] kinds(): ContainerKind[] // Events onUpdate(listener: NetworkUpdateListener): () void // Connection close(): Promisevoid }值得注意的源码增强点ContainerReference与ContainerConnection都实现了RequestHandler支持代理调用并额外提供castT(interfaceName?)类型化代理——将容器引用/连接直接转为强类型的服务接口调用interface MyService { sayHello(name: string): Promisestring } const containerRef await client.get(my-service, {}) const service containerRef.castMyService(MyService) const greeting await service.sayHello(Alice)通信模式Communication PatternsHuly Network 支持多种通信模式1. 请求/响应同步直接向容器发送请求并等待响应const result await containerRef.request(processData, { value: 42 }) console.log(result) // { processed: true, result: 84 }2. 即发即忘异步发送数据但不等待响应await containerRef.request(logEvent, { event: user_login, timestamp: Date.now() })3. 事件广播发布/订阅容器向所有已连接客户端广播事件// Client side const connection await containerRef.connect() connection.on async (event) { console.log(Received:, event) } // Container side connect(clientId: ClientUuid, broadcast: (data: any) Promisevoid): void { this.clients.set(clientId, broadcast) } // Broadcast to all clients for (const broadcast of this.clients.values()) { await broadcast({ type: update, data: changes }) }4. 双向流式传输建立持久连接进行流式通信const connection await containerRef.connect() // Receive stream connection.on async (chunk) { console.log(Chunk:, chunk) } // Send requests await connection.request(subscribe, { topic: updates }) await connection.request(getData, { range: [0, 100] })补充仓库中hcengineering/network-backrpc包backrpc/src提供了基于 ZeroMQ 的 RPC 通信层负责容器端点之间的双向消息通道request语义在底层可通过网络代理proxy转发也可在建立连接后走直连路径examples/README.md 中的 Example 6 演示了直连 vs 路由连接的性能差异首次连接后直连可绕过网络路由器。生命周期管理Lifecycle Management引用计数Reference Counting网络使用引用计数管理容器生命周期每次client.get()使引用计数 1每次containerRef.close()使引用计数 -1引用计数为 0 的容器在containerTimeout源码默认 5 秒内保持存活超时后容器被自动终止。// Acquire reference (ref count 1) const ref1 await client.get(service as any, { uuid: svc-1 }) // Acquire another reference to same container (ref count 2) const ref2 await client.get(service as any, { uuid: svc-1 }) // Release first reference (ref count 1) await ref1.close() // Release second reference (ref count 0) await ref2.close() // Container kept alive for containerTimeout, then terminated健康监控Agent 健康Agent 每pingInterval默认 1 秒必须 ping 一次网络在aliveTimeout默认 3 秒后将该 Agent 标记为死亡死亡 Agent 的容器会被移除。容器健康容器响应ping()调用ping 失败可能触发终止不健康的容器从注册表中移除。在源码 network.ts 中可以看到NetworkImpl内部维护了_orphanedContainers孤儿容器表周期性检查并清理无主容器这正是自动清理失败 Agent 与孤儿容器容错能力的实现基础。优雅关闭Graceful Shutdown// Container cleanup async terminate(): Promisevoid { // 1. Notify connected clients await this.broadcastShutdown() // 2. Close external connections await this.database.close() // 3. Clear internal state this.data.clear() // 4. Release resources this.connections.clear() } // Agent cleanup await agentServer.close() tickManager.stop() // Client cleanup await containerRef.close() await client.close() // Server cleanup await server.close() tickManager.stop()端点引用Endpoint References容器通过**端点引用endpoint references**被寻址。文档中以类型化字符串示意type ContainerEndpointRef string { _containerEndpointRef: true }端点类型直连端点Direct Endpoint直接连接到容器tcp://host:port/uuid路由端点Routed Endpoint通过 Agent 连接agent://host:port:agentId/uuid无连接端点No-Connect Endpoint仅请求、无持久连接noconnect://host:port/uuid源码对照上述 URI 形式是文档的示意性写法。在仓库实际实现 endpoints.ts 中端点引用以JSON 编码的EndpointRefData表示包含kindEndpointKind.routed | direct | noconnect、host、port、agentId与可选uuid字段parseEndpointRef内部即JSON.parse(ref)。agentDirectRef、agentNoConnectRef、containerDirectRef、containerOnAgentEndpointRef分别构造不同类型端点。解析端点import { parseEndpointRef, EndpointKind } from hcengineering/network-core const parsed parseEndpointRef(endpoint) console.log(parsed.kind) // EndpointKind.direct | routed | noconnect console.log(parsed.host) // Host address console.log(parsed.port) // Port number console.log(parsed.uuid) // Container UUID console.log(parsed.agentId) // Agent ID (for routed)容器种类与标签Container Kinds and Labels容器种类Kinds容器按kind字符串类型分类type ContainerKind string { _containerKind: true }常见示例user-session— 用户会话管理workspace— 工作区容器query-engine— 查询处理transactor— 事务处理Agent 声明其支持的种类生产环境使用serveAgent()// Note: For production code, use serveAgent() on the client // This example uses AgentImpl directly for educational purposes const agent new AgentImpl(agent-1 as any, { user-session: sessionFactory, workspace: workspaceFactory, query-engine: queryFactory })网络会汇总所有 Agent 支持的 kind 全集源码 network.ts 中kinds()将各 Agent 的 kinds 展平去重客户端可用client.kinds()做服务发现。标签Labels容器可携带labels用于细粒度选择// Create container with labels await client.get(workspace as any, { labels: [premium, us-west, production] }) // Labels enable: // - Multi-tenancy (tenant ID as label) // - Geographic routing (region labels) // - Tier-based selection (free, premium, enterprise) // - Environment separation (dev, staging, production)多租户场景的完整落地示例见 examples/03-multi-tenant.ts 与 MULTI_TENANT.md每个租户通过labels: [tenantId]获得彼此隔离的容器实例。GetOptionsinterface GetOptions { uuid?: ContainerUuid // Specific container UUID extra?: Recordstring, any // Additional parameters for factory labels?: string[] // Labels for selection }源码 api/types.ts 中该接口完全一致ContainerRecord额外记录了agentId、endpoint、lastVisit最近访问时间等运行时元数据配合NetworkEvent/NetworkEventKindadded0 / updated1 / removed2见 api/types.ts向客户端派发 Agent 与容器变化事件可用于监控与告警参考 examples/04-complete-production-setup.ts 中的onUpdate监控器示例。核心概念速览Network 中央协调者Agent 承载容器的工作节点Container 承载业务逻辑的服务实例Client 请求容器的应用Reference Counting 自动化生命周期管理Endpoint 到达容器的地址Kind 容器的类型/类别Labels 细粒度选择条件这些概念共同构成了基于 Huly Network 构建可扩展、容错分布式系统的基础。进一步学习路径容器开发指南 — 构建你的第一个容器快速上手 — 数分钟内跑通端到端示例无状态容器高可用 与 快速上手 HA — 配置自动故障转移多租户架构 — 基于标签的租户隔离生产环境部署 — 部署到生产环境自动回收指南 — 资源自动释放机制详解可运行示例见 foundations/net/examples/含基础请求/响应、事件广播、多租户、完整生产配置、错误处理与重试、自定义超时、HA 无状态容器等 7 个示例单元与集成测试覆盖见 packages/core/src/test含network.spec.ts、ha-stateless.spec.ts、alive-checkins.spec.ts等与 packages/server/src/test/network.spec.ts可作为理解行为契约的参考【免费下载链接】platformHuly — All-in-One Project Management Platform (alternative to Linear, Jira, Slack, Notion, Motion)项目地址: https://gitcode.com/GitHub_Trending/platform80/platform创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考