One of the risks of the software supply chain is secrets leaking. Secrets are all around the software supply chain; developers and the CI\CD pipelines need to use secrets to access the SCM, the pipeline, the artifact registries, the cloud environments, and external services. And when secrets are everywhere, it is a question of time until they leak.
This risk is well identified, and many software-supply-chain frameworks address it by requiring secret-scanning and secret-self-expiration.
So, what could go wrong if you, as a security practitioner, have put these guardrails in place?
This week, while researching secret scanning tools, I came across one answer. This answer is simple: since secret scanners search for patterns of secrets, when the secret format changes, detection may fail. And this is exactly what happened when GitHub introduced a new security feature – the fine-grained-tokens. This beta feature is one of GitHub’s ways to reduce secret-leaking risks; limiting the permissions of the personal access tokens also limits the risks had these secrets been compromised.
It turns out that the format of the new tokens is a bit different, while the old tokens’ format was something like:
ghp_123456789012345678901234567890123456
The new token format is like:
github_pat_123456789012345678901234567890123456789012345678901234567890…
Both the prefix and the length of the secret are different.
And indeed, some secret scanners fail to detect the new format;
To experiment, I created a repo with a Dockerfile with a secret and a workflow that runs the Trivy action. This is how the Dockerfile looks at the beginning of the experiment:
Following is a snapshot of the GitHub Secret Scanner, which detects the newly formatted secret:
As we can see, GitHub’s secret scanner detects the secret, but no alerts appear in the Code scanning section.
To verify that the tooling is set correctly, I will now add a classical secret to the Dockerfile (see below) and run the scanning again. Now we get a code scanning alert (only on the classic secret!)
Github’s scanner, on the other hand, now detects two secrets:
I have opened a security issue for Trivy; Trivy’s team responded that this is not a vulnerability nor a security issue. What was left to do was to open an issue.
This experiment raises many concerns;
- Why should a GitHub user suspect that the token format would be modified due to a new feature?
- Given that secrets are supposed to look like random strings, why should a GitHub user even care about the format of the secrets?
- Should GitHub be responsible for updating its customers on such a change?
- Is this secret-detection failure a vulnerability of GitHub’s fine-grained secrets, a vulnerability of Trivy, or, as Aqua Security sees it, not a vulnerability at all?
- Should Aqua Security, the company behind Trivy, be responsible for updating it?
- Since Trivy is an open source project, supplied as-is, would anyone be liable, had secrets leaked from a pipeline protected by Trivy? Who would it be? GitHub? Aqua Security? The Trivy user?
- What does this experiment teach us about trusting security tools installed to protect software supply chains?
I shall leave these questions open.
One thing is clear, though; protecting the software supply chain is complicated and we need a highly professional community to enduringly succeed in this task.
This post was co-authored by Avi Waxman, intern at Scribe Security
This content is brought to you by Scribe Security, a leading end-to-end software supply chain security solution provider – delivering state-of-the-art security to code artifacts and code development and delivery processes throughout the software supply chains. Learn more.
Related posts
