KarstenSamaschke
Back to all writing

Field notes

Trustworthy or Bustworthy

AI-driven supply-chain attacks undermine any trustworthiness. Here is how to handle that.

Karsten Samaschke · · 6 min read

Trustworthy or Bustworthy

The situation

I rely on pre-built packages every day. So does every developer on the planet. That approach broke in the last weeks with AI-driven supply-chain attacks spiking and poisoning many projects. I personally have been telling developers for years to compile from source.

Turned out I was right.

The new reality: Supply-chain attacks galore

In the past weeks, a lot of AI-augmented supply-chain attacks (also called "Shai Hulud" attacks) have been carried out, with the TrapDoor attack just being the latest.

This specific attack targeted crypto, DeFi, and AI tooling developers, harvesting wallets, SSH keys, AWS credentials, and GitHub tokens.

But it did not stop at stealing credentials. The payload planted poisoned .cursorrules and CLAUDE.md files using zero-width Unicode characters to hide malicious instructions inside them. The attackers submitted pull requests to projects like LangChain, LlamaIndex, and OpenHands to inject these files upstream.

The npm variant ran a postinstall hook with over 1,100 lines of credential harvesting code. The PyPI variant executed on import, downloading JavaScript from attacker-controlled GitHub Pages. The Rust variant used build.rs scripts to encrypt wallet keystores via XOR before sending them to GitHub Gists.

The trust void

With that in mind, and such kinds of attacks constantly going on nowadays, one cannot trust any kind of upstream binary source any: Not npm packages, not pip installs, not Cargo binaries, not pre-compiled libraries from a CDN, and not proprietary closed-source solutions either.

There is no reliable way of knowing whether these have been poisoned, whether the version you are installing is the one that was published, or whether a postinstall hook contains 1,149 lines of credential harvesting code disguised as a harmless utility, unless changing approaches.

Binary packages: Whitelisting only

For binary packages: Establish a whitelisting approach and treat it as a mandatory process.

Pinned, CVE-checked packages passing through a gate into a local registry, the only source your build system talks to

First, check every upstream package against CVE databases before you even consider pulling it. Only versions without CVE issues can be considered, yet have to be treated with care nonetheless. If a package has unpatched vulnerabilities, it does not enter the whitelist.

Second, only allow pinned versions - no ranges, no latest, no floating tags. Every allowed binary must be a specific version, vetted, and stored in your own repo or registry hub.

Third, maintain your own repository or registry hub. Never trust the upstream. That hub becomes the only source your build system talks to. If a dependency is not in the hub, it does not get installed.

Source Code: Build from scratch and audit

The rest of your dependencies - the hundreds you do not whitelist - follow a different path. You build them from source in a controlled environment after auditing the source code.

Building from source with individual components assembled instead of downloaded pre-compiled

The real attack vectors are in the build hooks - postinstall, preinstall, build.rs, setup.py with arbitrary Python execution, Makefiles that download remote payloads.

The first layer of verification is not reading the source. It is ensuring the build process is deterministic and reproducible. If you cannot rebuild the binary from the source in isolation and get the same output, you are trusting, not verifying.

AI as audit aid

But then: How to audit hundreds, thousands, tens of thousands lines of code?

You could do that manually.

Or, you could use AI to assist you.

The TrapDoor attackers (and many other AI-guided attackers alike) used them as a delivery mechanism.

The irony is: AI can also be part of the defense, if used correctly. A properly skilled and mechanically enforced verification workflow could use an AI assistant to analyze source code, postinstall hooks, build scripts, and unusual imports - flagging patterns that do not match what the package claims to do.

A simple prompt like "review this postinstall script and list every external network call, file write, and command execution" is a legitimate audit aid.

But the AI is a scanner, not an auditor. A human still needs to read the flagged items and decide whether they are legitimate or suspicious. The AI cannot tell you whether a function calling require('crypto') is supposed to hash a checksum or encrypt credentials for exfiltration. You still need domain knowledge for that.

There is no such thing as Vibe-Auditing.

Poisoned config files - and how to defend against them

What about prompt injection through poisoned config files?

Scanning a config file with a magnifying glass reveals contamination under scrutiny

.cursorrules, CLAUDE.md, .github/copilot-instructions.md - these files control how an AI assistant behaves. If an attacker plants one, the assistant follows instructions that were not written by the developer.

The defense is straightforward: Always treat these config files as untrusted input. Scan them for zero-width and control characters before any AI assistant reads them. Use a hex editor or cat -v on every config file that you did not write yourself. Do not let an AI assistant read a project directory without first inspecting the rules files inside it. The TrapDoor campaign worked because developers assumed config files were safe, but they are not anymore.

This applies to containers as well. A container built from a poisoned base image or with poisoned build dependencies carries the compromise into every deployment. Just saying.

Bottom line

If you are building software and cannot audit the source of your components, you are not building software. You are assembling someone else's code and hoping it does not bite you.

The approach that used to be called responsible engineering is now the only defense that stands between your codebase and a credential-stealing postinstall hook that gets picked up by your AI assistant and exfiltrates your SSH keys to a GitHub Gist.

The pain in adjusting will be real. But it is the fastest and most reliable way of mitigating supply-chain attacks once the supply chain itself can no longer be trusted.

Sources used