Blog
Horus now remembers the incident you already solved
Horus v0.24.3 finds the related past incident far more often at the start of an investigation, flags past incidents in the code a change touches, and ships IncidentBench, an open benchmark for incident recall.
Most production incidents are not new. The same queue backs up, the same supplier times out, the same error code comes back with a different wording in the alert. Whoever solved it last time already knows the first check to run. The problem is that the next investigation usually starts from zero.
Horus has saved every investigation as project memory since v0.12. With v0.24.3, it gets much better at bringing back the right one at the right moment: when a new investigation starts, and when someone is about to change the code an old incident lived in.
What changed
- Startup recall finds the related incident far more often. On six real codebases, scored on history the rules were never tuned on, Horus now surfaces a related earlier incident for 79–83% of repeat investigations, up from 63–67%, while staying quiet on 96% of incidents that are genuinely new.
- Past incidents in this code.
horus changesandhorus what-changednow list earlier incidents whose code the change modifies or removes, with the cause found at the time. - IncidentBench. The benchmark we used to measure all of this is open: the method, the case format, the runner and three public synthetic codebases.
Recall at the start of an investigation
Before Horus queries a single log, it checks what this repository has already been through. If an alert carries a fingerprint, error code, workflow or operation that matches a past incident, that match is direct. Most of the time, though, all Horus has is the hint someone typed or the alert text, and that wording rarely repeats exactly. The previous rule looked for word overlap and missed most rewordings.
The new matcher scores a hint against every wording each past incident was investigated under, not just the latest one. Two signals combine:
- Keyword coverage. Code-shaped identifiers such as
PAY_017orcreateRefundare kept exact, camelCase and snake_case names are split into words, and rare terms count for more than common ones. A rare identifier shared with only one or two past incidents is enough on its own. - Local embedding similarity. The source-intelligence host on your machine embeds each hint with the same local model it uses for code. Nothing leaves your machine. If the host is down, Horus falls back to keywords alone and still clears most repeats.
When something clears the bar, the investigation opens with up to three earlier incidents: what was found then, the first check to repeat, and the report to reopen. It is context, not a verdict. Horus still collects fresh evidence and never scores the new incident from the old one. In the synthetic payments codebase from IncidentBench, a later PAY_017 alert opens like this:
Prior incident 77d4b015-… [production; inferred]:
ProcessorClient.authorize calls queue behind an undici
Agent capped at connections=50.
Matched: errorCode, operation; last verified: never.
Reports: 77d4b015-…; historical outcome: unknown.
First check: Check current evidence for PAY_017 spikes.
Historical match; current outcome unconfirmed.How much better
Recall is only half the story. A tool that always shows something would find every repeat and be useless, so we report how often Horus stays quiet when nothing related exists:
| Set | Found | Stayed quiet on new incidents |
|---|---|---|
| Six real codebases (held out) | 79–83% | 96% |
| Six real codebases, keyword-only fallback | 72–74% | 96–97% |
| Public realistic | 297 / 375 | 84 / 85 |
| Public hard | 21 / 116 | 63 / 64 |
The hard track is there to show headroom, not quality. Its repeats are mostly adversarial rewordings and findings-only follow-ups that no hint matcher should be expected to catch.
Why we trust these numbers
- Nothing is tuned on the codebase being scored. Every threshold was chosen on the other codebases and then applied to the held-out one (leave-one-codebase-out).
- Two independent label sets. Reviewers grouped each real history into incident families without seeing any tool output. We report the range across both.
- Replayed in time order. Each step sees only the investigations that existed before it, exactly like the real thing.
- One rule for every repository. There is no per-project tuning to drift out of date.
Past incidents in the code you are changing
The same memory is useful before anything breaks. When a change modifies or removes a method that sat at the centre of an earlier incident, the reviewer should know. horus changes and horus what-changed now add a section for exactly that.
$ horus changes main HEAD
…
## Past incidents in this code
- 2026-07-02 PAY_017 spikes (77d4b015-…)
touches ProcessorClient.authorize; cause then: undici Agent
connections=50 too low for peak charge volumeAn incident counts only if Horus rated its top cause likely or highly likely, and only its top three method or function seeds form its footprint. We checked this against 409 real commits and 200 investigations: these links point at the code that was later changed for the incident about twice as often as chance.
Context, not a gate
The same data showed that touching an incident's code does not predict the next incident. So this section tells a reviewer which history to read, and nothing more: no risk score, no blocked merge. It shows the newest five matches, one per question, and it is included, redacted, when you push a change report to Horus Cloud.
IncidentBench
We could not find a public benchmark for this task, so we wrote one. IncidentBench v0 measures startup recall: each case is one codebase's investigation history, replayed in time order. For every investigation the tool is asked for related past incidents, using only what existed before that moment. Found and clean are reported together, because neither means anything alone.
The public cases are three synthetic codebases (payments, logistics and B2B SaaS) with ground truth defined when they were written, plus precomputed hint vectors so the embedding lane replays exactly without a model host. Results on private client codebases are never published. The method, case format and runner are in the horus-rust repository.
git clone https://github.com/Meritt-dev/horus-rust && cd horus-rust
HORUS_BENCH_CASES=bench/incident/cases/realistic \
cargo run -q -p horus-engine --example incident_benchIf you build incident tooling, we would like to see your numbers on it, and we would welcome more cases.
Upgrade
npm install -g @merittdev/horus@latest
# or
brew upgrade meritt-dev/tap/horus
# or
curl -fsSL https://horus.sh/install.sh | bashNothing to configure. The first investigation after upgrading indexes your existing memory for similarity search, spending at most two seconds on it, and every investigation after that adds its own hint. To give coding agents the updated instructions, run horus skill install claude --force (or cursor); an existing skill is not replaced without --force. For codex and gemini the skill lives in your AGENTS.md or GEMINI.md, which --force would overwrite whole, so copy the new text from horus skill print codex (or gemini) into the Horus section instead.