Blog

Horus now remembers the incident you already solved

Horus v0.24.3 finds the related past incident far more often at the start of an investigation, flags past incidents in the code a change touches, and ships IncidentBench, an open benchmark for incident recall.

October 25, 2026The Horus team6 min read

Most production incidents are not new. The same queue backs up, the same supplier times out, the same error code comes back with a different wording in the alert. Whoever solved it last time already knows the first check to run. The problem is that the next investigation usually starts from zero.

Horus has saved every investigation as project memory since v0.12. With v0.24.3, it gets much better at bringing back the right one at the right moment: when a new investigation starts, and when someone is about to change the code an old incident lived in.

What changed

  • Startup recall finds the related incident far more often. On six real codebases, scored on history the rules were never tuned on, Horus now surfaces a related earlier incident for 79–83% of repeat investigations, up from 63–67%, while staying quiet on 96% of incidents that are genuinely new.
  • Past incidents in this code. horus changes and horus what-changed now list earlier incidents whose code the change modifies or removes, with the cause found at the time.
  • IncidentBench. The benchmark we used to measure all of this is open: the method, the case format, the runner and three public synthetic codebases.

Recall at the start of an investigation

Before Horus queries a single log, it checks what this repository has already been through. If an alert carries a fingerprint, error code, workflow or operation that matches a past incident, that match is direct. Most of the time, though, all Horus has is the hint someone typed or the alert text, and that wording rarely repeats exactly. The previous rule looked for word overlap and missed most rewordings.

Diagram: a new investigation is checked against past incidents by structured alert fields and by hint match (keyword coverage plus local embedding similarity). Matches above the bar return the top three past incidents; otherwise Horus stays quiet.

The new matcher scores a hint against every wording each past incident was investigated under, not just the latest one. Two signals combine:

  • Keyword coverage. Code-shaped identifiers such as PAY_017 or createRefund are kept exact, camelCase and snake_case names are split into words, and rare terms count for more than common ones. A rare identifier shared with only one or two past incidents is enough on its own.
  • Local embedding similarity. The source-intelligence host on your machine embeds each hint with the same local model it uses for code. Nothing leaves your machine. If the host is down, Horus falls back to keywords alone and still clears most repeats.

When something clears the bar, the investigation opens with up to three earlier incidents: what was found then, the first check to repeat, and the report to reopen. It is context, not a verdict. Horus still collects fresh evidence and never scores the new incident from the old one. In the synthetic payments codebase from IncidentBench, a later PAY_017 alert opens like this:

text
Prior incident 77d4b015-… [production; inferred]:
  ProcessorClient.authorize calls queue behind an undici
  Agent capped at connections=50.
  Matched: errorCode, operation; last verified: never.
  Reports: 77d4b015-…; historical outcome: unknown.
  First check: Check current evidence for PAY_017 spikes.
  Historical match; current outcome unconfirmed.

How much better

Repeat investigations where the related incident was recalled (higher is better)Previous word-overlap ruleHorus v0.24.3
Six real codebases (held out)
Previous word-overlap rule: 63–67%
Horus v0.24.3: 79–83%
Public realistic
Previous word-overlap rule: 50.7%
Horus v0.24.3: 79.2%
Public hard
Previous word-overlap rule: 10.3%
Horus v0.24.3: 18.1%

Recall is only half the story. A tool that always shows something would find every repeat and be useless, so we report how often Horus stays quiet when nothing related exists:

SetFoundStayed quiet on new incidents
Six real codebases (held out)79–83%96%
Six real codebases, keyword-only fallback72–74%96–97%
Public realistic297 / 37584 / 85
Public hard21 / 11663 / 64

The hard track is there to show headroom, not quality. Its repeats are mostly adversarial rewordings and findings-only follow-ups that no hint matcher should be expected to catch.

Why we trust these numbers

  • Nothing is tuned on the codebase being scored. Every threshold was chosen on the other codebases and then applied to the held-out one (leave-one-codebase-out).
  • Two independent label sets. Reviewers grouped each real history into incident families without seeing any tool output. We report the range across both.
  • Replayed in time order. Each step sees only the investigations that existed before it, exactly like the real thing.
  • One rule for every repository. There is no per-project tuning to drift out of date.

Past incidents in the code you are changing

The same memory is useful before anything breaks. When a change modifies or removes a method that sat at the centre of an earlier incident, the reviewer should know. horus changes and horus what-changed now add a section for exactly that.

Diagram: the methods a change modifies or removes are intersected with the top code seeds of saved investigations whose cause was rated likely or highly likely, producing a Past incidents in this code section.
text
$ horus changes main HEAD
…
## Past incidents in this code
- 2026-07-02 PAY_017 spikes (77d4b015-…)
  touches ProcessorClient.authorize; cause then: undici Agent
  connections=50 too low for peak charge volume

An incident counts only if Horus rated its top cause likely or highly likely, and only its top three method or function seeds form its footprint. We checked this against 409 real commits and 200 investigations: these links point at the code that was later changed for the incident about twice as often as chance.

Context, not a gate

The same data showed that touching an incident's code does not predict the next incident. So this section tells a reviewer which history to read, and nothing more: no risk score, no blocked merge. It shows the newest five matches, one per question, and it is included, redacted, when you push a change report to Horus Cloud.

IncidentBench

We could not find a public benchmark for this task, so we wrote one. IncidentBench v0 measures startup recall: each case is one codebase's investigation history, replayed in time order. For every investigation the tool is asked for related past incidents, using only what existed before that moment. Found and clean are reported together, because neither means anything alone.

The public cases are three synthetic codebases (payments, logistics and B2B SaaS) with ground truth defined when they were written, plus precomputed hint vectors so the embedding lane replays exactly without a model host. Results on private client codebases are never published. The method, case format and runner are in the horus-rust repository.

bash
git clone https://github.com/Meritt-dev/horus-rust && cd horus-rust
HORUS_BENCH_CASES=bench/incident/cases/realistic \
  cargo run -q -p horus-engine --example incident_bench

If you build incident tooling, we would like to see your numbers on it, and we would welcome more cases.

Upgrade

bash
npm install -g @merittdev/horus@latest
# or
brew upgrade meritt-dev/tap/horus
# or
curl -fsSL https://horus.sh/install.sh | bash

Nothing to configure. The first investigation after upgrading indexes your existing memory for similarity search, spending at most two seconds on it, and every investigation after that adds its own hint. To give coding agents the updated instructions, run horus skill install claude --force (or cursor); an existing skill is not replaced without --force. For codex and gemini the skill lives in your AGENTS.md or GEMINI.md, which --force would overwrite whole, so copy the new text from horus skill print codex (or gemini) into the Horus section instead.

Give your AI agent grounded project memory