Investigation walkthrough
Follow a realistic incident from first alert to postmortem draft with Horus.
This tutorial walks through a complete incident investigation. You will initialize Horus for a repository, run an investigation for a checkout latency spike, review the report, replay it, and generate a postmortem. The commands are real; the output examples are realistic composites based on the actual CLI format.
Prerequisites
If you have not installed Horus yet, start with Getting started. You need a Git repository, and optionally source-intelligence and runtime connectors for the richest results. Investigation history is kept in an embedded local store at ~/.horus — nothing to install.
The scenario
You support a payments service called atlas-payments. An alert fires: checkout latency is spiking. Customers report timeouts at checkout. You do not know yet whether it is code, infrastructure, a downstream dependency, or a queue backlog. You will use Horus to build a ranked, evidence-backed picture of what happened.
Step 1: Initialize the project
Change into the repository and run horus init. One command does the whole onboarding: it checks prerequisites (the optional source-intelligence backend, advisory only), creates .horus/config.json, registers atlas-payments in the global registry, starts or reuses the source-intelligence host, indexes the code, stitches the queue map — what lets Horus connect queue producers to workers across async boundaries — and refreshes the project-knowledge snapshot.
cd /path/to/atlas-payments
horus init --env productionIf you already have a source-intelligence host running, pass its URL with --source. Horus records it verbatim and skips the local host spawn and index.
horus init --env production --source http://127.0.0.1:8420Re-running horus init later is idempotent: it reuses a healthy host and refreshes the index. You can confirm host health with horus hosts.
horus hostsStep 2: Run the investigation
Start from the hint exactly as it arrived in the alert.
horus investigate --env production "checkout latency spike"Horus performs several steps behind the scenes:
- Resolves the hint to relevant symbols in the codebase, such as
CheckoutControllerorPaymentGateway. - Collects runtime evidence from any configured connectors: Elasticsearch/Axiom logs, Sentry errors, Grafana metrics, MongoDB/Postgres state, Shopify Admin data, and Redis/BullMQ queue state.
- Pulls recent git changes around the suspected symbols.
- Scores suspected causes based on the evidence.
- Ranks the causes and prints a report.
Reading the output
A typical report is organized into several sections, the most important being:
- Evidence — a flat list of every evidence item, each tagged
[source/kind](code, logs, metrics, state, queue, history). Queue signals also get a dedicated Queue runtime block and git changes a Recent changes block. - Suspected causes (ranked) — ranked hypotheses with confidence scores, each linking back to supporting/contradicting evidence. Horus is honest about confidence: scores are capped when the report rests on weak or structural-only evidence, the header warns with a '⚠ low-confidence closest match' disclaimer when no symbol closely matched your hint, and it prints 'No dominant suspected cause emerged' instead of inventing a culprit. Data-state findings are phrased as '(co-occurring) … verify whether it relates', and dependency/network failures (e.g. getaddrinfo ENOTFOUND) are synthesized from error/queue forensics and promoted as the headline cause.
- Evidence gaps — sources that are not configured or did not return relevant data. Gaps are not failures; they tell you where more connectors might help.
- Suggested next — a short, deterministic list of follow-up commands that fit this result. Horus classifies the hint first: a live symptom like "latency spike" runs as an incident with full runtime evidence, while a structural hint like "what depends on PaymentGateway" runs as a source-impact question and routes you to
horus blast-radius. Horus only suggests; it never runs the follow-up for you. See Self-routing.
AI narrative is optional
Add --ai to get an AI-generated explanation of the deterministic report. The AI only summarizes the evidence; it does not create new evidence. If the AI provider is unavailable, Horus falls back to the deterministic report.
Step 3: Dig deeper with source intelligence
Suppose the top suspected cause is PaymentGateway.charge. You can ask Horus to explain the symbol and estimate blast radius before deciding where to look next.
horus explain PaymentGateway.charge --depth 2
horus blast-radius PaymentGateway.charge --depth 2horus explain shows callers, callees, and file locations. horus blast-radius shows upstream and downstream impact, including queue consumers. These commands are read-only and do not change system state.
If you need to find an owner, run:
horus owner PaymentGateway --jsonStep 4: Replay the investigation
Every investigation is saved to your local store at ~/.horus. List recent investigations to get the ID of the one you just ran.
horus investigations -n 5Replay re-renders the saved report without re-querying your production systems. This is useful for sharing, reviewing, or formatting the output differently.
horus replay <id> --format markdownReplay vs re-run
horus replay uses the evidence already collected. horus investigate collects fresh evidence. Replay is faster and safe to share; re-run is what you want when the incident is still evolving or you have fixed a connector.
Step 5: Draft a postmortem
Once you understand the incident, generate an editable postmortem draft from the saved investigation.
horus postmortem <id> --output ./postmortem-checkout-latency.md --forceThe draft includes a summary, impact, timeline, contributing factors, a candidate root cause, an evidence list, lessons learned, and follow-up actions. Treat it as a starting point: confirm the actual root cause and fill in the action items before publishing it.
Human review required
A generated postmortem is not the final incident report. It is a structured draft that saves you time. Always review and edit it.
What you learned
In this walkthrough you:
- Initialized a project with
horus init— prerequisite checks, config, host startup, indexing, and the queue map in one command. - Ran a deterministic investigation from a hint.
- Used
horus explain,horus blast-radius, andhorus ownerto dig deeper. - Replayed a saved investigation without re-querying systems.
- Drafted a postmortem from the investigation.
From here, you can add runtime connectors to fill evidence gaps, explore the CLI reference for every command, or read the Source intelligence and Connectors pages to understand what is happening under the hood.