1. Tenant and scope validationverifiedGuardrails · 06:41:12

   Passport verified for ag-kubernetes-01. Scope limited to tenant tn-nordic / customer cu-fsprod. Read-only mode confirmed.

   • passport signature valid
   • tenant boundary check passed
2. Node status queryanomalyKubernetes · 06:41:38

   fs-prod-cs-tool2 reports Ready=False, kubelet heartbeat stale for 4m12s.

   • kubectl get node fs-prod-cs-tool2 -> NotReady
   • kubelet last heartbeat 06:37:26
3. Node conditionsanomalyKubernetes · 06:42:02

   MemoryPressure=False, DiskPressure=False, PIDPressure=False, NetworkUnavailable=False, Ready=False (KubeletNotReady: container runtime network not ready).

   • conditions snapshot captured
4. Kubernetes eventsanomalyKubernetes · 06:42:31

   17 FailedCreatePodSandBox events and 9 Failed ErrImagePull events on the node within 10 minutes.

   • event stream 06:32-06:42
5. Kubelet logsanomalyLinux · 06:43:04

   kubelet: failed to pull image registry.corp.internal/cni/calico-node:v3.27.2 — connection reset by peer during layer fetch.

   • journalctl -u kubelet (read-only)
6. Containerd logsanomalyLinux · 06:43:29

   containerd: 3 resets mid-transfer at ~5MB layer boundary; TLS handshake succeeds, stream terminates.

   • journalctl -u containerd (read-only)
7. Prometheus metricsinfoObservability · 06:44:10

   node CPU 21%, memory 48%, disk 39%, no saturation. container_runtime_operations_errors_total rising for PullImage only.

   • PromQL snapshot
8. Image pull error correlationanomalyEvidence · 06:44:51

   All failures target the external registry path; internal mirror pulls on the same node succeed.

   • 9/9 failures external registry
9. Network evidenceanomalyNetwork · 06:45:33

   Outbound TCP 443 to registry egress IP resets after 5-8 seconds. Egress path traverses an SSL-inspection appliance added in change CHG-20482.

   • egress probe (read-only)
   • CHG-20482 change window matches onset
10. Hypothesis: node resource exhaustionrejectedReasoning · 06:46:02

    Rejected — no pressure conditions, utilisation well under thresholds.

11. Hypothesis: kubelet certificate expiryrejectedReasoning · 06:46:20

    Rejected — client certificate valid until 2026-12-03, API server auth succeeding.

12. Hypothesis: CNI configuration driftrejectedReasoning · 06:46:44

    Rejected — CNI config identical to healthy peer nodes; failure is at image fetch, not config parse.

13. Hypothesis: registry egress / SSL inspection resetverifiedReasoning · 06:47:15

    Supported by all four evidence classes. Confidence 88%.

14. Final RCA compiledverifiedRCA · 06:48:02

    Read-only RCA issued with recommendation. No production write performed.

    • rca-inc-4821

Final root cause

Confidence 88% · risk low · no production write required