How should we manage credential redaction?

[IsmaelMartinez]

redact_credentials (git_provider.py:159) and the artifact redaction proposed in #3989 match credentials with hand-written patterns, so anything outside the list passes through, and each new format means another regex. Should we keep growing our own patterns, or adopt a maintained detector with a curated rule set plus entropy checks? detect-secrets (Yelp, pure Python) is the obvious candidate, though its last release was May 2024. presidio is maintained but aimed at personal data and heavy. Any switch adds a runtime dependency, so views welcome before anyone opens a PR.

[IlyaKhalafi]

I would put a detector behind the existing redaction interface, then compare it with the current rules before replacing them. Pin the detector version so it can be replaced behind that interface. detect-secrets is a detector rather than a drop-in string redactor.

Two integration details matter. Its baseline JSON stores a hash and line metadata, not replacement spans. Live findings are per line and retain the matched value, so replacement needs to happen in memory before serializing the result. Its plugins and filters also target source-code scanning. For outgoing artifacts, I would explicitly choose the filters and omit the verification filter, which can send candidate credentials to provider APIs. Do not inherit baseline exclusions or let an allowlist comment decide what is safe to publish.

Configure plugins and filters once at startup, and check that a synthetic canary is detected. Defaults can leave the plugin list empty. Settings are process-global, and scan_line() mutates them. Avoid per-request transient_settings; consume list(scan_line(line)) inside the serialization lock.

Keep structural handling for URL userinfo, Authorization headers and known credential fields, since those can contain low entropy values. Use the detector to extend coverage for bare tokens.

Before switching, run both against a synthetic corpus containing those existing cases, multiline values, escaped strings and benign high entropy identifiers. Compare missed redactions, damaged non-secret text and latency on realistically sized artifacts. Neither entropy nor a curated catalogue establishes that arbitrary output is secret-free.

[IsmaelMartinez]

Thanks @IlyaKhalafi, this is the detail the question needed, especially the verification filter and the process-global settings. Agreed on the shape: keep the structural handling for URL userinfo, Authorization headers and known credential fields, and add a detector only for bare tokens, behind redact_credentials. As that adds a runtime dependency, the next step is the comparison you describe on a synthetic corpus before any PR; would you be up for running it?

[IlyaKhalafi]

Yes. I ran a first comparison against c7b2793 with detect-secrets==1.5.0. The script and results use synthetic data only.

Signature detectors caught Stripe, Twilio and SendGrid examples that the current rules missed. Entropy checks caught more opaque values, but also masked benign hashes and IDs. Every detector filter was disabled, including heuristics, so this is a starting point for tuning.

For a 64 KiB synthetic log, median processing times were about 9 ms for the current redactor, 66 ms with signature detectors added, and 293 ms with entropy added too. These are five warm runs on one machine, excluding initialization and artifact I/O.

Two integration details matter. scan_line() needs an explicit entropy threshold check on eager candidates. Its JWT finding can also leave the signature segment visible when used for replacement. Verification never ran and network connections were blocked.

I would keep the structural redactor and start with a narrow signature-only profile.