ARCHITECTURE DOSSIER · 07

日志、可观测与恢复

Nginx、systemd、Docker、Loki、Grafana、备份与故障演练

RISKOPSCODE + RUNTIME + QA
OPS · COMPONENT TOPOLOGYowner / runtime / truth state
01Client
Synthetic / operatormixed

外部探针、追踪查询、告警响应

QA + SREexternal + browser
02Edge
Nginx access/errorrunning

request_id、route、status、latency

Platform OpsEC2 files
03Service
systemd / Docker logsrunning

structured event、domain ids 与 errors

Service ownersjournald / json logs
Alloy → Loki → Grafanarunning

采集、索引、查询与 dashboard

Platform OpsDocker
Backup / restore jobsrisk

manifest、copy、verify、drill

Platform Ops + data ownerstimers/scripts
04State
Single root EBSrisk

当前 DB、repos、packages、logs、backups

Platform Opsone failure domain
每次跨层调用必须同时具备IDENTITYVERSIONED CONTRACTTELEMETRYFAILURE POLICY

CRITICAL SEQUENCE

真实主链与事务边界

每一步都必须能回到调用方、契约、状态 owner 与失败证据;UI 只显示服务端已经确认的真值。

  1. 01

    Edge 接收请求并生成/透传 request_id

  2. 02

    服务日志绑定 actor/domain entity/release

  3. 03

    Alloy 收集 Nginx、Docker 与 journals

  4. 04

    Loki 按 service/release/status 查询

  5. 05

    Grafana 告警关联 runbook 与 owner

  6. 06

    备份离机、校验并定期在隔离环境恢复

INTERFACE REGISTER

跨项目接口与安全条件

producer、consumer、契约和 guard 同时登记;其中任何一列缺失都不能进入多团队并行开发。

ProducerVersioned contractConsumerSecurity / consistency guard
NginxAccess event {request_id,host,path,status,duration}Loki

PII redaction

ServicesStructured log {level,event,entity,release,trace}Alloy/Loki

no secrets / bounded labels

Backup jobBackupManifest {source,as_of,hash,target}Restore drill

off-host copy + verify

STATE OWNERSHIP

数据真值与恢复

LogsPlatform Ops

STORELoki + journal

RETENTION当前约 4 天本机

RECOVERY不作为业务恢复源

DB dumps / package copiesData owners

STORElocal backup now

RETENTIONtimer policy

RECOVERY需迁离根卷

Gitea reposDeveloper Platform

STORErepos + PostgreSQL

RETENTIONsnapshot timer

RECOVERY需要隔离 restore drill

FAILURE MODEL

故障必须怎样收敛

TRIGGER根卷损坏

SYMPTOM业务、日志与备份同时丢失

CONTAIN当前无跨盘隔离

P0 architecture finding
TRIGGER日志标签爆炸

SYMPTOMLoki 写入/查询退化

CONTAIN高基数字段放 body,不做 label

ingestion metrics
TRIGGER备份文件存在但不可恢复

SYMPTOM灾难时无法启动服务

CONTAIN以 restore drill 作为唯一通过标准

drill report + RTO

QA RELEASE GATES

不是画完架构图就算完成

这些门禁连接 UI 状态、接口负向路径、运行证据与可恢复性;发布证据必须绑定 exact commit。

OPS-01

request_id 贯穿边缘与服务

synthetic trace

OPS-02

日志不含 token/PII

redaction scan

OPS-03

异地副本与 hash 校验

P0 · 未完成

OPS-04

按服务演练 RPO/RTO

季度 restore drill