Investigate · workspace

P2investigating

SCADA edge cluster losing telemetry batches

Edge collectors drop 4% of telemetry batches during peak windows.

Evidence viewerOpen RCA report

Elapsed

146h 37m 06s

live clock

Reference

inc-4809

recurrence 4x

Investigation workspace

Evidence-backed timeline — every step is read-only, bounded and audited.

Read-only Agent OS

Status

investigating

Tenant

Helios Energy Grid

SCADA Edge Estate

Lead agent

Network Agent 01

Environment: production

RCA confidence

pending

Investigation timeline

Severity P2 · phases are append-only evidence steps

  1. Tenant and scope validationverifiedGuardrails · 06:41:12

    Passport verified for ag-kubernetes-01. Scope limited to tenant tn-nordic / customer cu-fsprod. Read-only mode confirmed.

    • passport signature valid
    • tenant boundary check passed
  2. Node status queryanomalyKubernetes · 06:41:38

    fs-prod-cs-tool2 reports Ready=False, kubelet heartbeat stale for 4m12s.

    • kubectl get node fs-prod-cs-tool2 -> NotReady
    • kubelet last heartbeat 06:37:26
  3. Node conditionsanomalyKubernetes · 06:42:02

    MemoryPressure=False, DiskPressure=False, PIDPressure=False, NetworkUnavailable=False, Ready=False (KubeletNotReady: container runtime network not ready).

    • conditions snapshot captured
  4. Kubernetes eventsanomalyKubernetes · 06:42:31

    17 FailedCreatePodSandBox events and 9 Failed ErrImagePull events on the node within 10 minutes.

    • event stream 06:32-06:42
  5. Kubelet logsanomalyLinux · 06:43:04

    kubelet: failed to pull image registry.corp.internal/cni/calico-node:v3.27.2 — connection reset by peer during layer fetch.

    • journalctl -u kubelet (read-only)
  6. Containerd logsanomalyLinux · 06:43:29

    containerd: 3 resets mid-transfer at ~5MB layer boundary; TLS handshake succeeds, stream terminates.

    • journalctl -u containerd (read-only)

Opened 8/1/2026, 9:55:00 PM · inc-4809