Betterleaks secrets scanner: architecture and keys
The detection of secrets in repositories has changed considerably in recent years. Previously, it was enough to look for suspicious strings or keys with high entropy in the code. Today, the situation is different: larger repositories, faster CI/CD pipelines, and, above all, an increasing amount of code generated by automated tools or AI models.
This has a practical consequence: the problem is no longer just finding secrets, but separating what is truly dangerous from what merely appears to be. Many teams are discovering that the real cost of these scanners lies not in running the analysis, but in reviewing hundreds of false positives.
Detection architecture: what changes with Betterleaks
Betterleaks appears precisely in this context. It doesn't attempt to completely reinvent secret scanning, but it does challenge a widespread assumption: that detecting patterns is enough.
In many modern repositories it is not.
The project, developed by Zach Rice and maintained with support from Aikido, proposes something slightly different. Instead of focusing solely on detecting matches, it attempts to validate whether the finding makes sense before escalating it as an alert.
This might seem like a minor detail, but it significantly changes the dynamics in large teams. When a scanning system generates too many irrelevant alerts, the team's natural reaction is to ignore them. And in security, an ignored alert can be worse than no alert at all.
To address this problem, Betterleaks introduces two interesting technical pieces: validation using CEL (Common Expression Language) and a metric called “Token Efficiency”, based on BPE tokenization.
The idea is that not everything that appears to be a secret actually is. Some high-entropy strings are simply hashes, identifiers, or automatically generated fragments. The system's goal is to reduce that noise.
The project documentation mentions a comparison where BPE tokenization achieves a 98.6% recall rate compared to the 70.4% obtained using entropy in the CredData dataset. As with any benchmark, these numbers are indicative. They serve well as a reference point, but do not replace testing in real repositories.
Source: GitHub
Components that make the difference
Reviewing the project's characteristics reveals a clear direction: to facilitate deployment in real-world environments without adding too much technical complexity.
Among the most prominent elements are:
- Rule-defined validation using CEL (Common Expression Language)
- Token Efficiency Scanning based on BPE tokenization rather than entropy, achieving 98.6% recall vs 70.4% with entropy on the CredData dataset
- Pure Go implementation (no CGO or Hyperscan dependency)
- Automatic handling of doubly/triply encoded secrets
- Expanded rule set for more providers
- Parallelized Git scanning for faster repository analysis
Although this list may seem like just a set of technical improvements, what's interesting is how they affect everyday use.
For example, a full Go implementation with no native dependencies greatly simplifies integration into CI/CD pipelines. In many teams, small details like that determine whether a tool ends up being used or gets forgotten in a repository.
BPE tokenization also introduces a different approach. Instead of simply measuring the randomness of a chain, it analyzes token patterns that more closely reflect how modern credentials are actually structured.
What happens when the scanner finds something?
When Betterleaks detects a potential secret, the process doesn't end there.
First, the context is evaluated using rules defined in CEL. This allows for the addition of further conditions: for example, checking if the format matches the expected provider or discarding patterns that frequently appear in examples or fictitious data.
This step may seem trivial, but it has a significant practical impact. False positives not only waste time but also reduce the team's confidence in the alert system.
Another interesting aspect is the automatic handling of secrets encoded multiple times. In some repositories, credentials appear transformed using base64 or other encoding schemes, which complicates their detection.
Even so, it's worth remembering something that's sometimes overlooked: no scanner can completely replace human review. Detecting a secret is just the beginning; deciding what to do with it (revoke, rotate, ignore, or investigate) remains a contextual decision.
Governance and human-centered approach/AI
Betterleaks is published under the MIT license and features external contributions from organizations such as Royal Bank of Canada, Red Hat, and Amazon.
The project also attempts to adapt to a reality that is increasingly visible in modern repositories: the mix of code written by developers and code generated by automated tools.
In this context, the tool aims to function well in both human-operated workflows and automated systems that review entire repositories. This aligns with the growing use of automation and tools that analyze code or generate automatic reviews.
The roadmap also includes interesting ideas: integration with data sources beyond GitLanguage model assistance for classifying findings and automatic revocation mechanisms via provider APIs.
This opens up an interesting debate. Automating credential revocation can reduce the time it takes to respond to an incident, but it also means relying on the classification system to be accurate.
If an automatic revocation fails or is triggered by mistake, the operational impact can be considerable.
Practical implications and limitations
From an operational point of view, Betterleaks is attractive to teams looking to reduce false positives and simplify deployments.
But it's also important to keep some limits in mind:
- Recall metrics depend on the dataset used and can vary considerably between repositories.
- Automating actions such as key revocation requires additional controls and audit logs.
- Secret scanners remain just one layer of defense within a broader strategy.
In many cases, the decision to adopt such a tool depends not so much on its theoretical accuracy as on something simpler: whether it integrates well into the team's workflow.
A highly accurate scanner that generates too much friction is usually abandoned. A reasonably accurate one that is easy to integrate is usually retained.
In that sense, Betterleaks attempts to strike a balance. It doesn't promise to eliminate all false positives or replace existing security processes, but it does aim to reduce noise and facilitate integration into modern pipelines.
The project is available on GitHub and is presented as an evolution of the approach used by Gitleaks, with the intention of adapting to repositories where automation, analysis agents and code generated by language models are a regular part of the development flow.
