Dapr 监控仪表盘实战指南基于 Grafana 模板监控系统服务、Sidecar 与 Actor【免费下载链接】daprDapr is a portable runtime for building distributed applications across cloud and edge, combining event-driven architecture with workflow orchestration.项目地址: https://gitcode.com/GitHub_Trending/da/dapr导读本文以 Dapr 仓库 grafana 目录下的三套官方 Grafana Dashboard 模板为核心系统讲解如何在 Kubernetes 集群中通过 Prometheus 采集 Dapr 指标并导入仪表盘实现对控制平面系统服务dapr-operator、sidecar-injector、sentry、placement、Dapr Sidecar运行时与Actor 运行时的吞吐、延迟、资源与安全状态的立体化监控。读完本文你将掌握每个仪表盘的面板构成、核心 PromQL 查询背后的指标语义、模板变量的用法以及指标在源码中的定义位置能够独立完成 Dapr 可观测性的落地与排查。一、仓库中的监控资产总览Dapr 仓库在 grafana 目录下提供了三份可直接导入 Grafana 的仪表盘 JSON 模板模板文件仓库相对路径仪表盘名称监控对象核心关注点grafana/grafana-system-services-dashboard.jsonDapr System Services DashboardDapr 控制平面系统服务dapr-operator、dapr-sidecar-injector、dapr-sentry、dapr-placement 的健康与资源、注入请求、证书签发等grafana/grafana-sidecar-dashboard.jsonDapr Sidecar DashboardDapr Sidecardaprd 运行时Sidecar 健康/资源、HTTP 与 gRPC 的吞吐和延迟、Actor、mTLS、组件状态grafana/grafana-actor-dashboard.jsonDapr Actor DashboardDapr Actor 运行时Actor 调用吞吐/延迟、Timer/Reminder 触发、基于单线程模型的并发与锁等待三份模板的核心语义与原始文档一致且与 docs/development/dapr-metrics.md 中定义的指标一一对应构成指标采集 → PromQL 查询 → 可视化面板的完整链路。README 中提到的 Supported Dapr metrics 对应仓库路径为 docs/development/dapr-metrics.md是阅读本指南时配合查阅的指标权威清单。二、前置条件让 Prometheus 能采集到 Dapr 指标Grafana 仪表盘本身只是可视化层数据来自 Prometheus。Dapr 的所有进程控制平面各服务与 sidecar默认都会启动一个 Prometheus 格式的 metrics HTTP 端点其默认配置定义在 pkg/metrics/options.go 中const ( defaultMetricsPort 9090 defaultMetricsAddress 0.0.0.0 defaultMetricsEnabled true )即默认监听 0.0.0.0:9090默认开启。每个 Dapr 进程都可通过--metrics-port命令行参数覆盖端口从 Helm Chart 模板如 charts/dapr/charts/dapr_operator/templates/dapr_operator_deployment.yaml、charts/dapr/charts/dapr_placement/templates/dapr_placement_statefulset.yaml、charts/dapr/charts/dapr_sentry/templates/dapr_sentry_deployment.yaml 等可以看到operator、placement、scheduler、sentry、sidecar-injector 均通过--metrics-port显式注入该配置。部署 Prometheus 采集时只需让 Prometheus 的 ServiceMonitor 或prometheus.io/scrape注解覆盖这些 Pod 的:9090/metrics端点即可。采集到数据后在 Grafana 中添加一个名为Dapr的 Prometheus 数据源三份模板的面板目标均使用datasource: Dapr或datasource: null继承默认数据源就可以导入模板了。三、Dapr System Services Dashboard控制平面健康与安全模板 grafana/grafana-system-services-dashboard.json 的description字段自述为 This dashboard shows the metrics of Dapr control plane services即监控 Dapr 控制平面四个关键服务dapr-operator、dapr-sidecar-injector、dapr-sentry、dapr-placement。JSON 中version: 7表明该模板已经过多轮迭代。3.1 Health Resource健康与资源该行面板组以app标签过滤四个控制平面服务核心查询包括Uptimetime() - max(process_start_time_seconds{app~dapr-sentry|dapr-placement-server|dapr-sidecar-injector|dapr-operator}) by (app)用当前时间减去进程启动时间以秒为单位展示每个服务已稳定运行时长。Total CPU usage基于container_cpu_usage_seconds_total按 Pod 匹配计算 CPU 使用量。Heap Memory usage / Number of GO routines直接使用 Go 运行时指标go_memstats_alloc_bytes与go_goroutines。Dapr 默认启用 Prometheus 的 process collector 与 go collector见 docs/development/dapr-metrics.md 中process_*、go_*指标说明因此这些指标无需额外配置即可获得。3.2 Placement成员与 Actor 注册面板组包含Number of connected Dapr sidecars与actor sidecars vs non-actor sidecars两个面板对应查询dapr_placement_runtimes_total向 placement 服务上报的主机sidecar总数。dapr_placement_actor_runtimes_total上报的 Actor 运行时数量。dapr_placement_runtimes_total - dapr_placement_actor_runtimes_total两者之差即非 Actor 的 sidecar 数量。这些指标定义于 pkg/placement/monitoring/metrics.go同文件还包含dapr_placement_actor_heartbeat_timestampActor 心跳时间戳、dapr_placement_leader_status与dapr_placement_raft_leader_status1 为 leader、0 为 follower可用于判断 placement 主从切换。3.3 Sidecar Injector注入请求监控 Webhook 注入的请求量dapr_injector_sidecar_injection_requests_total# sidecar injection requestsdapr_injector_sidecar_injection_succeeded_total# successful sidecar injected对应实现位于 pkg/injector/service/metrics.go。当注入失败率上升时应重点检查 MutatingWebhook 配置、镜像拉取与 init 容器执行情况。3.4 CA Sentry证书与 mTLS 安全围绕 pkg/sentry/monitoring/metrics.go 定义的证书签发指标构建包括Certificate Signing Requests (CSR) from Daprdsum(dapr_sentry_cert_sign_request_received_total)收到的 CSR 总数。CSR Failuressum(dapr_sentry_cert_sign_failure_total)签发失败数。Server TLS certificate issuance failuressum(dapr_sentry_servercert_issue_failed_total)服务端 TLS 证书签发失败数。Issuer cert and key changed totalsum(dapr_sentry_issuercert_changed_total)颁发者证书轮换次数。Root/Issuer cert expirymin(dapr_sentry_issuercert_expiry_timestamp) - time()颁发者证书剩余有效时间秒用于提前发现证书过期风险。四、Dapr Sidecar DashboardSidecar 运行时全景模板 grafana/grafana-sidecar-dashboard.jsonversion: 8面向每个应用 Pod 注入的 daprd 运行时标签体系为service格式{app_id}-dapr与app_id。其面板行结构与 README 描述的sidecar health/resources、throughput/latency of HTTP and gRPC、Actor、mTLS完全对应。4.1 Health monitor健康监控Sidecar uptimetime() - max(process_start_time_seconds{service~($dapr_app_id)-dapr, namespace$namespace}) by (service)面板阈值设定为600 秒——若 Sidecar 运行不足 10 分钟面板显示为红色快速暴露频繁重启CrashLoopBackOff / OOMKilled的实例。4.2 Resource usage资源使用复用 Go/Prometheus 通用指标sum(rate(container_cpu_usage_seconds_total{pod~...}))内核态 用户态 CPU 总用量。sum(go_memstats_alloc_bytes{service~...})堆内存占用。sum(process_virtual_memory_bytes{service~...})虚拟内存占用。sum(go_goroutines{service~...})Goroutine 数量可辅助诊断协程泄漏。4.3 Throughput/Latency - HTTP 与 gRPC该部分分别覆盖OutboundApp → Dapr与InboundDapr → App两个方向每个方向展示 RPS 与 95p/75p 分位延迟查询模式如下HTTP 入站延迟Dapr → Apphistogram_quantile(0.95, sum(rate(dapr_http_server_latency_bucket{app_id~$dapr_app_id, namespace$namespace}[5m])) by (le, app_id, method, path))HTTP 出站延迟App → Dapr基于dapr_http_client_roundtrip_latency_bucket。HTTP RPSsum by (app_id, method, path) (rate(dapr_http_server_response_count{...}[5m]))与sum by (app_id, method, path) (rate(dapr_http_client_completed_count{...}[5m]))。gRPC 同理基于dapr_grpc_io_server_server_latency_bucket、dapr_grpc_io_client_roundtrip_latency_bucket与dapr_grpc_io_server_completed_rpcs、dapr_grpc_io_client_completed_rpcs。这些 HTTP 指标定义在 pkg/diagnostics/http_monitoring.gogRPC 指标则由 OpenCensus ocgrpc 插件生成详见 docs/development/dapr-metrics.md 的 gRPC monitoring 一节。注意 HTTP 指标以斜杠/前缀标记dapr_http_server_request_count、dapr_http_server_response_count、dapr_http_server_latency与dapr_http_client_roundtrip_latency、dapr_http_client_completed_count等。4.4 Components组件健康对应运行时组件生命周期指标见 pkg/diagnostics/service_monitoring.goLoaded component configurationssum(dapr_runtime_component_loaded{app_id~...})Initialized componentssum(dapr_runtime_component_init_total{app_id~...})Component failuressum(dapr_runtime_component_init_fail_total{app_id~...})组件初始化失败数持续上升时可直接定位到具体app_id对应的 state store、pubsub 或 binding 配置问题。4.5 Actor运行时 Actor 状态Status report status / Report status error reasonsrate(dapr_runtime_actor_status_report_total{...})与rate(dapr_runtime_actor_status_report_fail_total{...})反映 sidecar 向 placement 上报 Actor 状态的成败。Placement table operationsum(dapr_runtime_actor_table_operation_recv_total{...})。Actor table rebalancingdapr_runtime_actor_rebalanced_totalActor 分片再平衡次数。Actor Deactivationdapr_runtime_actor_deactivated_total与dapr_runtime_actor_deactivated_failed_total可用于发现 Actor 无法正常停用的异常。4.6 SecuritymTLS 安全对应运行时安全指标pkg/diagnostics/service_monitoring.gomTLS initialization status / mTLS Initialization Errorsrate(dapr_runtime_mtls_init_total{...})与rate(dapr_runtime_mtls_init_fail_total{...})。Workload certificate rotation status / Cert rotation error reasonsrate(dapr_runtime_mtls_workload_cert_rotated_total{...})与rate(dapr_runtime_mtls_workload_cert_rotated_fail_total{...})。证书轮换失败往往与 sentry 可达性、SPIFFE ID 配置或时钟偏差相关可与系统服务仪表盘的 CA Sentry 面板联动排查。五、Dapr Actor DashboardActor 调用的吞吐、延迟与并发模板 grafana/grafana-actor-dashboard.jsonversion: 3聚焦 Actor 模型其描述为 This is the dashboard for Dapr Actor。5.1 Throughput/latency吞吐与延迟面板通过path正则/v1.0/actors/$dapr_actor_type/.*精确过滤 Actor HTTP API 流量区分两个方向Actor API LatencyApp → Dapr95p/75phistogram_quantile(0.95, sum(rate(dapr_http_server_latency_bucket{path~/v1.0/actors/$dapr_actor_type/.*, namespace$namespace}[5m])) by (le, app_id, method, path))该查询可同时观察到每个 statestore 与 service invocation 在 Actor 路径上的往返性能。Actor API request rateApp → DaprRPS基于dapr_http_server_response_count按path匹配累加。Actor Callback latency / request rateDapr → App基于dapr_http_client_roundtrip_latency_bucket与dapr_http_client_completed_count反映 Dapr 调用用户业务方法Actor 方法实现的耗时与频率。5.2 Timer Reminder定时器与提醒面板行Timer Reminder提供dapr_runtime_actor_timers_fired_totalActor timer 触发次数与dapr_runtime_actor_reminders_fired_totalActor reminder 触发次数的可视化。这两项指标与 pkg/diagnostics/service_monitoring.go 中dapr_runtime_actor_timers、dapr_runtime_actor_reminders等系列一起可用来评估定时任务的触发频率是否符合预期以及排查 timer/reminder 丢失或延迟问题。5.3 Concurrency并发与锁等待Dapr Actor 采用基于 Actor 的 turn-based 并发模型每个 Actor 实例同一时刻只处理一个调用。面板Concurrency使用dapr_runtime_actor_pending_actor_calls{actor_type~$dapr_actor_type}Pending locks Per Actor展示等待获取 per-Actor 锁的排队调用数。该值持续偏高通常意味着某个 Actor 实例被慢方法长时间占用是定位 Actor 热点与性能瓶颈的关键信号。六、模板变量多租户与多应用筛选三份仪表盘通过 Grafana 模板变量实现灵活筛选模板变量定义于各 JSON 的templating段仪表盘变量类型与查询作用System Services无查询型变量—以app标签硬编码过滤四个控制平面服务Sidecarnamespacelabel_values(dapr_runtime_component_loaded,namespace)按命名空间筛选Sidecardapr_app_idlabel_values(dapr_runtime_component_loaded,app_id)按应用 ID 筛选当前默认值为 loadtestclient、stateactor 等测试应用Actornamespacelabel_values(dapr_runtime_component_loaded,namespace)按命名空间筛选默认pipelineActordapr_actor_typelabel_values(dapr_runtime_actor_pending_actor_calls,actor_type)按 Actor 类型筛选默认StateActor变量查询都基于dapr_runtime_component_loaded指标从源码结构看这是因为该指标在所有 sidecar 上稳定存在且带namespace、app_id标签天然适合作为变量枚举源。导入模板后建议根据实际集群环境调整变量的默认值。七、指标全景从仪表盘到源码定义的对照仪表盘中的每个查询都对应 docs/development/dapr-metrics.md 中登记的指标除健康类指标外统一以dapr_前缀开头。按来源划分指标族典型指标源码位置Operatordapr_operator_service_created_total/deleted_total/updated_totalpkg/operator/monitoring/metrics.goSidecar-injectordapr_injector_sidecar_injection_requests_total/succeeded_total/failed_totalpkg/injector/service/metrics.goPlacementdapr_placement_runtimes_total、dapr_placement_actorruntimes_total、dapr_placement_leader_status等pkg/placement/monitoring/metrics.goSentrydapr_sentry_cert_sign_*、dapr_sentry_issuercert_expiry_timestamp等pkg/sentry/monitoring/metrics.goSchedulerdapr_scheduler_sidecars_connected、dapr_scheduler_jobs_triggered_total等pkg/scheduler/monitoring/metrics.goRuntime 服务类dapr_runtime_component_loaded、dapr_runtime_service_invocation_*、dapr_runtime_mtls_*、dapr_runtime_actor_*pkg/diagnostics/service_monitoring.goRuntime 错误码error_code_countpkg/diagnostics/errorcode_monitoring.goRuntime HTTPdapr_http_server_*、dapr_http_client_*客户端指标含 error bodypkg/diagnostics/http_monitoring.goRuntime gRPCdapr_grpc_io_server_*、dapr_grpc_io_client_*由 OpenCensus ocgrpc 生成参见 docs/development/dapr-metrics.md gRPC 节Workflowdapr_runtime_workflow_*、dapr_runtime_workflow_activity_*pkg/diagnostics/workflow_monitoring.go组件pubsub/binding/state/config/secret/jobdapr_component_pubsub_ingress_latencies、dapr_component_state_latencies等pkg/diagnostics/component_monitoring.go以 pkg/operator/monitoring/metrics.go 为例可以印证指标的定义方式源码通过 OpenCensus 的stats.Int64(operator/service_created_total, ...)声明指标再注册为 Prometheus 视图导出Prometheus 抓取时会在命名空间前缀下暴露为dapr_operator_service_created_total。理解这层OpenCensus 指标 → Prometheus 视图的映射有助于在仪表盘查询中准确书写指标名与标签如app_id、namespace、actor_type、method、path。另外值得留意的是docs/development/dapr-metrics.md 中还登记了 Scheduler 服务的指标dapr_scheduler_jobs_triggered_total、dapr_scheduler_trigger_latency等用于补充模板中尚未单独成盘的 Job/定时任务监控读者可按需自行扩展面板。八、快速上手指南准备 Prometheus确保集群内 Prometheus 已配置抓取 Dapr 各服务与 sidecar 的:9090/metrics端点默认端口见 pkg/metrics/options.go。添加 Grafana 数据源创建名为Dapr的 Prometheus 数据源指向 Prometheus 服务地址。导入模板在 Grafana 中依次导入 grafana/grafana-system-services-dashboard.json、grafana/grafana-sidecar-dashboard.json、grafana/grafana-actor-dashboard.json 三份文件或复制其 JSON 内容。配置模板变量在 Sidecar 与 Actor 仪表盘顶部选择namespace、dapr_app_id或dapr_actor_type聚焦目标命名空间与应用。建立监控习惯日常巡检建议按系统服务健康证书过期/注入失败→ Sidecar 资源与组件失败 → Actor 延迟与并发锁等待的顺序排查指标含义不明时回查 docs/development/dapr-metrics.md 获取逐项说明。三份模板覆盖了 Dapr 从控制平面到运行时、从通用 HTTP/gRPC 调用到 Actor 专用语义的完整监控面配合 docs/development/dapr-metrics.md 的指标清单与 pkg/diagnostics 等源码目录即可构建一套可解释、可排查的 Dapr 生产可观测体系。【免费下载链接】daprDapr is a portable runtime for building distributed applications across cloud and edge, combining event-driven architecture with workflow orchestration.项目地址: https://gitcode.com/GitHub_Trending/da/dapr创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考