Microsoft studies show poisoned training data and checkpoints can hijack coding agents

Microsoft Defender Research and the Microsoft AI Red Team published two studies on post-training supply chain attacks against AI coding agents on Nov. 12, 2026.

A backdoor contains hidden behavior that stays inactive until a trigger appears. Once activated inside an agent, the model may invoke tools, run commands, change files, or reach data. Coding and terminal agents look through repositories, read files, call APIs, and run commands, often reaching sensitive development environments. That activity can affect the surrounding system even while the agent seems to finish the user’s benign task.

Both attack paths abuse trust in post-training artifacts because large training datasets can hide risky examples and model weights defy source-code-style review. The entry points include victim-controlled training data or attacker-trained checkpoints before distribution. The attacker cannot pick which victim, harness, or runtime eventually uses the backdoored model. The hidden behavior stays dormant until the proper condition triggers it.

Dataset poisoning inserts a few crafted examples into fine-tuning material later used by the victim. Legitimate trajectories carrying those examples teach the model to identify a command-execution tool and execute an unsafe action when an ordinary user request arrives. A trigger can be a routine request from fine-tuning, or content found in a filename, code comment, or tool result. A compromised model can pick an unsafe action while defenses observing the input only see something benign.

An attacker might release a tainted dataset on a public hub by impersonating an organization, reusing a namespace, or taking over a trusted account. Another route places examples in public forums or scraped sources so bad rows slip into a derived dataset indirectly. Threat hunting across open-source hubs revealed some rows in a large dataset linking benign requests to unrelated, unsafe tool calls. One redacted example teaches the model to send a tool call with curl and source to pull and run a remote script from GitHub.

Researchers could not identify what the publisher intended or say whether those few rows would materially change a fine-tuned model.

Microsoft Defender Research studied how an attacker might taint supervised fine-tuning data used to specialize a coding or terminal agent. Researchers added 90 tainted examples to about 9,000 terminal-agent conversations, roughly 1% of the corpus, and spread the examples across 10 simulated harnesses using different tool names and schemas for command execution. The trigger was a routine request to create project documentation, chosen because users routinely make that request of coding agents, so an attacker needs no further involvement after poisoning. The model learned to look at available tools and pick one able to run commands, then formed a valid tool call with an unsafe command pulling and running content from an attacker-controlled domain, after which the attacker can wait for victims’ agents to phone their command-and-control server.

On two training harnesses, 500 trials after fine-tuning gpt-4.1-mini produced exact or equivalent unsafe actions at 78.0% and 77.8%. Transfer reached one unseen harness at 11.0%, while a second unseen harness stayed at 0%. Ten paraphrases of documentation requests yielded 52% to 70% attack success on seen harnesses and moved to the first held-out harness. Exact-phrase matching cannot catch this kind of conceptual generalization.

The research also described two backdoor designs. A fixed-payload backdoor trains the model to ask for a set action once a trigger appears. A data-derived backdoor trains the model to pull a parameter from runtime context and use it in an unsafe tool call without new poisoning. Researchers found no evidence of the data-derived backdoor method in the wild.

Model poisoning presumes an adversary controls training before a distributor ships an open-weight checkpoint, as with checkpoints circulated through model hubs. Model poisoning uses supervised fine-tuning and reinforcement learning to make backdoors more reliable without hurting general performance much. The attacker runs training and procedure, then ships a checkpoint without knowing its adopter, the coding harness, or the runtime permissions. The Microsoft AI Red Team checked whether a poisoned open-weight checkpoint could deliver an agent backdoor to downstream deployments.

Researchers trained backdoored coding models using trigger-bearing trajectories paired with clean counterfactuals that kept the task but dropped the trigger and unsafe action. Evaluation covered unseen repositories, different agent harnesses, and extra reinforcement learning. Backdoors worked outside the training setting and stayed effective after later training restored much benign performance. The checkpoint study fine-tuned Qwen3.5-4B and Qwen3.5-9B on about 2,400 solved coding trajectories.

Researchers matched poisoned examples with clean counterfactuals that kept the task but dropped the trigger and unsafe action. Evaluation used familiar tasks and five permissively licensed repositories not involved in building the poisoned trajectories. Benign cover tasks guided the agent to discover triggers via directory listings, file reads, or searches. When triggered, the model generated a properly formatted tool call that the harness accepted as legitimate.

Tests included actions harming availability, confidentiality, and integrity. Across both models and tested action categories, in-distribution attack success ranged from 85.9% to 100%. A basic transfer of the attack managed only 32.7% to 63.4% attack success. The strongest configuration reached 84.1% to 94.8% backdoor activation on out-of-distribution data.

False triggers occurred rarely in the aggregate results.

The gap between familiar and unseen settings indicates in-distribution evaluation may overstate backdoor robustness. Transfer tests used backdoored checkpoints on harnesses unseen during training. When triggered, the model fitted the unsafe action around each harness’s tool interface, and the harness executed it. Transfer success differed across harnesses mainly because of tool interfaces and invocation formats.

For Claude Code, attack success was 97.0% for the 4B model and 91.0% for the 9B model. For Pi, attack success reached 74.0% for the 4B model and 77.0% for the 9B model. For the Codex command-line interface (CLI), attack success came to 37.3% for the 4B model and 63.0% for the 9B model. Nothing in the results indicates a flaw in default Claude Code or Codex CLI configurations.

Risk appears only when a harness runs a backdoored model checkpoint. Deployment evaluations ought to use the production harness and its real tool schemas. Reasoning traces mattered in this experimental setup. Dropping trigger and action reasoning cut unseen-repository success from 94.8% to 1.0% for the 4B model, and from 84.1% to 0% for the 9B model.

Supervised fine-tuning hurt benign coding task performance for small models. After supervised fine-tuning, a short reinforcement-learning phase made the coding-agent backdoor more reliable. Reinforcement learning restored much utility with attack success unchanged, and the team observed improved performance on coding benchmarks. Agent backdoors turn hidden model behavior into actions that touch code, data, credentials, and networks.

A backdoored model tied to privileged tools can reach network resources, messages, credentials, data, files, and code. The tools available, together with their permissions, set the size of the possible blast radius. Once a model connects to tools, hidden behavior can reach credentials, code, data, or network resources.

Backdoor behavior generalizes past exact trigger strings, familiar repositories, and the training harness, with triggers appearing in ordinary user requests or context from repository files and tool results. Satya Nadella argued in Models as Insider Risks in the Super Intelligence Era that trustworthy enterprise AI requires independent observability and controls outside the model.

Reviewers should match the user request against the assistant action, since a well-formatted example can still be unsafe when its action is unrelated to or riskier than the request. Agent security relies on treating post-training artifacts as supply-chain items. Agent backdoors turn model-integrity failures into system actions.