The detection that was switched off in March

Issue one. A control that existed and had been disabled nine weeks earlier, a signup flow that failed silently for anyone behind a VPN, and the retention number that decides whether any of the rest matters.

ATT&CK
T1490

One thing that worked, one that did not, and one number worth watching. Short on purpose: a format that survives a busy week is worth more than one that produces an excellent issue every other month.

What worked

Writing down why a detection was suppressed.

While putting together the Kesterline report, the detail that kept coming back was not the intrusion. It was that the control which would have caught it existed, was correct, and had been set to audit-only nine weeks earlier after a false positive from a backup agent. Nobody could recall doing it. Nothing brought it back.

That is not a story about a careless team. Turning off a noisy rule at 2am is a reasonable emergency action, and the person who did it made the right call with the information they had. The failure was that the decision left no trace, so it could never be revisited.

The fix is unglamorous and it works: every suppression gets an owner and an expiry date. When the date passes, the rule comes back on its own unless somebody actively renews it. The default becomes protection rather than silence, and no one has to remember anything.

If you take one thing from this issue, take that. It costs an afternoon.

What did not

Our own signup, for anyone behind a VPN or a privacy extension.

The bot check on this site is Cloudflare Turnstile, and it is enforced server-side, which is correct. What was not correct is what happened when it failed: the reader was shown the API's own words.

captcha protection: request disallowed (invalid-input-response)

Somebody trying to create an account was told that, and nothing else. It reads as a fault in the site, gives them nothing to do, and the button they needed stayed disabled with no explanation of why. We found it because somebody tried to sign up on a phone whose browser blocks challenges.cloudflare.com, which is a configuration a meaningful share of this audience runs deliberately.

Three things were wrong and all three are now fixed. Errors are translated into what happened and what to try. The disabled button says what it is waiting for. And a twelve-second watchdog covers the case the original code missed entirely: Turnstile loading, rendering, and then never resolving, which is what a distrusted VPN egress actually produces rather than a clean failure to load.

The general lesson is the one worth keeping. A third-party control on your critical path needs a defined failure mode, and "the API's error string" is not one. We had thought about the check being blocked. We had not thought about it being blocked silently.

The number

Effective log retention, read against your own time to detect.

Not retention on its own, which everybody quotes from their most generous source. The number is the weakest source an investigation actually needs, held next to how long you typically take to notice something.

The arithmetic is usually uncomfortable. Windows Security logs default to 20MB, which on a busy domain controller can be under four hours. Mandiant's most recent M-Trends puts global median dwell time at 14 days. If both numbers describe your estate, the majority of your investigations begin after the authentication evidence has already rolled.

That comparison justifies more budget than any dashboard, and it costs nothing to work out. Raising the log size is a group policy change.

The full entry, including how it gets gamed, is on the metrics page.


Observations goes out on Fridays. It is called that because a lesson is not learned until behaviour has changed and somebody has checked; until then it has only been observed. Disagreements are the useful part, and there is a discussion under every issue.

Corrections and additions are welcome: this is a working document. Get in touch, or post a case of your own in the community.

Discussion

Guidelines

Sign in to comment. Corrections and additions are the point: this is a working document.