The findings we can publish, and the ones we can't.
A live finding published is a map. So this post is the method, and the findings we are free to publish — which are the ones we raised against ourselves.
Pattern one: the claim runs ahead of the wiring
The most reliable finding in any system that has been shipping for a while is not a vulnerability. It is a sentence. Somewhere in the documentation, the sales deck or the product page there is a capability described in the present tense that exists in the codebase but is not wired into the running deployment, or is behind a flag that is off, or worked once and is now broken. Nobody lied. A feature merged and a feature shipped are different events, and the copy tends to be written on the first one.
Our defence against it is dull and it works: the capability document that describes what Butler does also carries an explicit do-not-claim list — things that are built but not wired, flagged off, or broken. It exists so that anyone writing an external sentence has somewhere to check before writing it, and so that a feature has to be removed from a list, by a person, before it can be described as shipped.
The audit test is one question asked of every external claim: name the code path and the flag state that make this true in the deployment that is running right now. Claims that cannot survive it get rewritten, not quietly kept.
Our own published wording once described a unified audit log. Put to the test above, the phrase promised more than the running implementation delivers: what the system actually guarantees is that approvals are attributed to the operator who made them.
We changed the sentence rather than defend it, because the claim has to be true on the day somebody reads it, not on the day the roadmap catches up. That is the whole of the remedy for this pattern: correcting copy is cheap, and a buyer who checks one claim and finds it exact stops checking. A buyer who checks one and finds it stretched starts.
Pattern two: audit the writes, not the features
A feature-by-feature review of a mature operations system does not terminate. Butler is roughly 450 HTTP routes across 36 routers, with 17 external connectors and 13 shipped automations — counts of the codebase, from the project's internal capability document for the PPC International deployment, not outcomes of any kind. Nobody is reviewing 450 routes carefully, and a review that pretends to is worse than one that admits its scope.
The tractable question is narrower: enumerate every path that writes outside the system's own database, and for each one record three things.
Which credential
Not which account it is supposed to use — which one it actually presents, and what that credential is scoped to do.
Idempotent or not
If running it twice produces two of something with money or stock attached, that is a finding regardless of how unlikely the second run seems.
Behaviour on partial failure
What state the business is left in when the call fails halfway. A path that fails closed is reviewable; one that improvises is not.
That list is short even when the route list is not, because most routes read. Restricting the write paths on purpose — one destination per fact, everything else a reader — is what makes this kind of audit finishable at all, which is an argument for the architecture as much as for the method.
Pattern three: the deployment is part of the system
Reading a repository tells you what the code does. It does not tell you how many copies of it are currently running, which credentials each copy holds, or which scheduled jobs are enabled on which one. Those are deployment facts, they are not in version control, and an audit that stops at the code will not see them.
We know this because it is how we caught ourselves out. In July 2026 a staging copy of Butler was still holding production messaging credentials and spent nine days posting duplicate end-of-day drafts into a real channel. The code was correct in every stack. The configuration was wrong in one of them, and no amount of reading the repository would have shown it.
So the audit asks for three inventories that most teams have never written down in one place: the list of stacks actually running, the mapping of which secret is installed in which stack, and the list of scheduled jobs enabled per stack. Producing those three documents finds problems on its own, before anyone reviews them.
Pattern four: numbers without a method
Operations projects accumulate improvement figures — percentages of work automated, effort reduced, time saved. Traced back, a surprising share of them turn out to be somebody's estimate from one deployment, repeated until it acquired the texture of a measurement. There is no fraud in this and it is very hard to stop, because the number is usually roughly right and nobody wants to be the person who deletes it.
The test is the same four questions every time: what does it measure, over what period, at which deployment, and by what method. A figure that cannot answer all four is an impression, and it should be labelled as one or dropped. We apply this to our own material and it costs us numbers we would like to quote.
“One deployment is not an industry benchmark, and we would rather say so than round it up into a claim.”
The same discipline applies to certifications, where the pressure to hedge is strongest. The published position of ERP Automate — also a Brdy Holdings portfolio company — is that it does not hold a certification today and will not claim one until an audit is complete. Brdy AI claims none either. “Aligned with”, “built to the standard of” and “audit-ready” are all ways of writing a certification you do not have, and an auditor reads them as exactly that.
What we will not publish
We will not name a client system, publish a finding before it is fixed, or publish an aggregate detailed enough that somebody could work out which system it came from. That rules out most of what would make this post more interesting, and it is the same rule that will protect your findings if you engage us.
It is also worth saying plainly what the rule is not doing here. Brdy AI's deployments to date are in a business we own; there is no body of external client findings being discreetly withheld. We would rather write that sentence than let the phrase “client confidentiality” do work it has not earned.