Observations

What went wrong, and what to do about it

Real breaches, each put through the same four questions: one control that worked, one that did not, one number worth watching, and what should have been done instead. Every entry cites a primary source, because the point of this section is that you can take it to a meeting.

NO. 28/Technology/27 September 2026

One service account password in a personal Google profile, and 134 customers' support files

Okta Security, 'Unauthorized Access to Okta's Support Case Management System: Root Cause and Remediation', 3 November 2023

Not yet reviewed
This one was written here from the source and has not yet been signed off, so it carries no byline. Every factual claim in it comes from the document linked below and can be checked against it. The judgements are ours, and until somebody has been back through them they are marked as unverified rather than presented as settled.

From 28 September to 17 October 2023 a threat actor used a service account to reach files in Okta's customer support system associated with 134 customers. Some were HAR files containing session tokens, which the actor used to hijack the Okta sessions of 5 customers. Okta found that an employee had signed in to a personal Google profile in Chrome on an Okta-managed laptop, and the service account's username and password had been saved into that personal Google account, which Okta considers the most likely route of exposure.

What worked

Customers raised the alarm: 1Password reported suspicious activity to Okta Support on 29 September and BeyondTrust on 2 October. The turning point was an IP address supplied by BeyondTrust on 13 October. With it, Okta identified the service account on 16 October, disabled it and terminated its sessions on 17 October, and revoked the session tokens embedded in the HAR files.

What failed

For 14 days of active investigation Okta did not identify the suspicious downloads in its own logs. Its analysts looked at access to support cases, but the actor went straight to the Files tab of the support system, which generates an entirely different log event with a different record ID. The service account's credential was stored in the support system itself and carried permission to view and update customer support cases. Blocking personal Google profiles in Chrome on Okta-managed laptops was a remediation applied only afterwards. A gap in the support system logs also hid the final hours of the actor's access, so further downloaded files were only found on 19 October.

The number

14 days

of active investigation in which Okta did not identify the suspicious downloads in its own logs.

What to do instead

Block personal browser profiles and password sync on managed devices through enterprise browser policy, rather than trusting staff not to sign in. Keep service account secrets in a vault, scope them narrowly, and alert on any use from an unfamiliar network. Before an incident, list every log event type your SaaS vendors emit for file access and test that your hunt queries catch each route to the same data. Treat HAR files as credentials: strip session tokens before they are uploaded and keep token lifetimes short. When two customers report the same oddity within days, assume the common factor is you.

Named for the doctrine distinction between a lesson observed and a lesson learned: a lesson is not learned until behaviour has changed and somebody has checked. Pretending otherwise is how the same finding turns up in three consecutive post-incident reports.