Pillar C: Cybersecurity of AI SystemsC3

AI Supply Chain Security

Model provenance, dataset poisoning, Hugging Face risks, ML library vulnerabilities, trojanized models.

Part of Pillar C: Cybersecurity of AI Systems · Cybersecurity of AI Systems groups the disciplines that share methods, tools, and threat models with AI Supply Chain Security.

What is AI Supply Chain Security?

AI supply chain security addresses the risks introduced when organizations consume pre-trained models, datasets, and ML libraries from external sources. Just as traditional software supply chains became a prime attack vector (SolarWinds, Log4Shell), the AI supply chain presents analogous — and in some ways more dangerous — risks because models are opaque binaries that can embed backdoors invisible to code review.

Model provenance is a critical challenge. When a team downloads a model from Hugging Face, they inherit every risk from that model's training process — poisoned training data, embedded backdoors, malicious serialization payloads (pickle deserialization attacks are rampant), and undisclosed biases. The Hugging Face ecosystem alone hosts over a million models, many with minimal vetting. Researchers have demonstrated that malicious models can execute arbitrary code upon loading through Python's pickle format.

Dataset poisoning in the supply chain is equally concerning. Popular datasets like LAION, Common Crawl, and curated benchmark sets can be manipulated at scale. Attackers can contribute poisoned examples to public datasets, compromise data pipelines, or create convincing fake datasets that introduce subtle backdoors. Defending the AI supply chain requires model signing, provenance tracking, dependency scanning, and runtime integrity verification.

Why it matters

Organizations building on open-source models and public datasets inherit every upstream risk. Without supply chain security controls, a poisoned model from Hugging Face or a backdoored dataset can compromise an entire AI deployment.

AI supply chain security connects traditional software supply chain disciplines (SBOMs, dependency management, code signing) to the unique challenges of ML artifacts — models, datasets, and training pipelines — where traditional scanning tools are blind.

Standards and frameworks

Curated resources

Authoritative sources we ground AI Supply Chain Security questions in — frameworks, research, guides, and tools.

OWASPframework

OWASP Machine Learning Security Top 10

Top 10 security risks specific to machine learning systems, including supply chain attacks, data poisoning, and model theft.

Hugging Faceguide

Hugging Face Security Documentation

Security best practices for using Hugging Face Hub — model scanning, SafeTensors, access controls, and supply chain considerations.

Trail of Bitsguide

Trail of Bits — "AI/ML Security Auditing" research

Security audit firm with deep AI/ML expertise. Published research on pickle deserialization attacks, model file format security, and ML pipeline vulnerabilities. Technical depth from a security-first perspective.

Unknownresearch

Gu et al. — "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain" (2019)

Seminal backdoor attack paper. Demonstrated trojaned models in transfer learning scenarios. Foundational for AI supply chain security questions.

Hugging Facetool

Hugging Face — Model Security and Safety

The largest model hub. Security features: malware scanning, pickle scanning, safetensors format. Questions on model provenance, serialization risks (pickle exploits), and model marketplace trust.

JFrogguide

JFrog — "Malicious Models on Hugging Face" research

Discovered 100+ malicious models on Hugging Face exploiting pickle deserialization for code execution. Real-world evidence of AI supply chain attacks. Good for scenario-based questions.

NTIA / OpenSSFframework

SBOM for AI/ML — AI BOM (Bill of Materials)

Extending software bill of materials concepts to AI: model cards, data cards, training provenance. Emerging standard for AI supply chain transparency.

ProtectAItool

ProtectAI — AI/ML Vulnerability Database (huntr)

Bug bounty platform focused on AI/ML vulnerabilities. Real-world vulnerability data in ML frameworks and models. Good for grounding tool security questions in actual discovered vulnerabilities.

Hugging Faceguide

SafeTensors Documentation

Hugging Face's safe serialization format for ML models. Prevents arbitrary code execution from pickle-based attacks.

OpenSSFframework

SLSA — Supply-chain Levels for Software Artifacts

Framework for ensuring the integrity of software artifacts throughout the supply chain. Applicable to ML model pipelines.

OpenSSFtool

Sigstore — Software Supply Chain Security

Open-source project for signing, verifying, and protecting software supply chains. Keyless signing for artifacts.

Education and certifications

More in Cybersecurity of AI Systems

See how your AI Supply Chain Security skills stack up

300 questions available. Compete head-to-head or run a quick speed quiz to benchmark yourself.