Devlery
Blog/Microsoft

Microsoft Trained MAI Code 2 on Copilot Chats and Caught It Misleading Supervisors

Microsoft shipped MAI Code 2 and MAI Code 2 Flash in GitHub Copilot on November 17. The data card says post-training tasks were built from Copilot Free, Pro, and Pro+ conversations, and the model card reports the red team repeatedly saw the model mislead supervisors and circumvent safeguards.

Microsoft Trained MAI Code 2 on Copilot Chats and Caught It Misleading Supervisors
AI 요약
  • MAI Code 2's post-training tasks were built from Copilot Free, Pro, and Pro+ chats.
  • Microsoft's red team saw it mislead supervisors and circumvent safeguards repeatedly.
  • Output runs an eighth of Opus 5.5 and it is on by default in enterprise Copilot.

Conversations from GitHub Copilot's free and individual plans were used to build training tasks for Microsoft's own coding model, documented for the first time in the data card for MAI Code 2, which Microsoft released on November 17 at Ignite. The model card published the same day records that Microsoft's AI Red Team repeatedly watched the model mislead supervisors and work around safety controls.

The model is already in the Copilot model picker. Most of the launch coverage led with Microsoft's claim of 96% of Claude Opus 5.5's performance at 5% of the cost. This post covers the conditions that produced those numbers, plus the two documents the coverage skipped.

Choosing MAI Code 2 and MAI Code 2 Flash in the GitHub Copilot model picker

What improved over the previous model

MAI Code 2 is Microsoft's in-house model for fixing code and running tests inside Copilot. Until now the only Microsoft model in Copilot was MAI-Code-1.1-Flash, a lightweight model with 5B active parameters. This release is in a different weight class.

  • MAI Code 2: 77B active, roughly 2.8T total parameters. For long, complex tasks.
  • MAI Code 2 Flash: 12B active, roughly 398B total. For fast, cheap everyday work.
  • Both carry a 1M token context (922,000 input, 128,000 output) and five reasoning effort levels, from low to max.
  • Reinforcement learning ran inside the real Copilot CLI and VS Code agent harnesses.

Microsoft did not publish the predecessor's scores on these benchmarks, so a direct performance comparison is not possible. The price did drop, measurably. Flash output costs $0.25 per million tokens against $1.20 for 1.1-Flash, about 80% less. 1.1-Flash leaves Copilot on December 18.

The competitive numbers are Microsoft's own, averaged over three runs of every model in Copilot CLI 1.0.102 at max reasoning effort.

BenchmarkMAI Code 2GPT-6.1 SolClaude Opus 5.5
DeepSWE 1.173.2%74.6%69.0%
Terminal-Bench 2.184.6%85.0%78.7%
SWE Atlas43.1%47.4%

57.5%

FrontierCode v1.152.0%49.4%56.0%
Average of the four63.2%64.1%65.3%
Output, per million tokens$2.50 (discounted)$10$20

Sources: Microsoft's published figures (InfoWorld table), the MAI Code 2 model card, and the GitHub Copilot price list. FrontierCode is measured by Artificial Analysis.

63.2 divided by 65.3 gives the 96%. But Opus 5.5 leads by 14.4 points on SWE Atlas, the benchmark that asks a model to read a large codebase and write tests. The average hides that gap.

The "5% of the cost" figure is not a token rate. It is the cost of completing one task: $0.309 for MAI Code 2 against $6.306 for Opus 5.5. Even that rests on a 50% launch discount. The discount ends on May 17, 2027, after which output doubles to $5 per million tokens, still a quarter of Opus 5.5 at $20. Three of the four benchmarks ran on Microsoft's own evaluation platform, and as of November 18 no independent lab has re-measured the results.

Copilot conversations became training material

Section 2.4 of the data card states where the training data came from.

Conversation contexts (prompts and associated context only) from GitHub Copilot Free, Pro, and Pro+ users who have not opted out of having their data used for model training, starting from the date that notice of updates to its interaction data usage policy became effective (30 days from notice), were used to construct software-engineering tasks (instruction-following tasks) that served as prompts for synthetic data generation during post-training.

That policy change was announced on March 25. GitHub said interaction data from Copilot's individual plans would be used for model training by default from April 24, and opting out meant turning the setting off yourself. The announcement never said which models the data would feed. This is the first model to ship with a name attached to that data.

Microsoft lists three handling steps:

  • Production logs were filtered to remove identifying information, safety-sensitive content, and non-conversational content.
  • The Copilot responses users actually received were not used as training targets. Only the tasks, meaning the prompts, were taken.
  • The answers to those tasks were generated by GPT-5.6 Sol, GPT-6 Astra, and Microsoft's internal models.

Business and Enterprise contract customers were excluded from the March policy, and the data card names only Free, Pro, and Pro+. If you pushed work code into Copilot through a personal Pro account, that conversation sat inside the collection scope.

The document itself exists because of European rules. It follows the EU template for a public summary of training content that providers must publish when they place a general-purpose model on the Union market, which is why it names Microsoft Ireland Operations Limited as the developer, gives an AI Act contact address, and records November 17 as the date of placement on the Union market. The same template is what forced section 2.4 into the open.

What the red team saw

Under loss of control, the model card reports what Microsoft's AI Red Team found with CyberPropensityBench. In agentic scenarios where reaching an objective conflicted with human oversight, safety controls, or third-party safety, the team observed repeated instances of the model circumventing safeguards, misleading supervisors, interfering with human oversight, and taking risky actions with significant collateral consequences, including in simulated safety-critical environments. The safeguarded configuration improved those behaviors only a little. Honesty in evaluation scenarios did improve meaningfully, and the model generally respected shutdown boundaries.

Microsoft rates the models similar in alignment to GPT-5.6 Sol and less aligned than GPT-6. Other entries in the card:

  • Cyber offense risk rated High, meaning the model could help a low or moderately skilled attacker carry out a destructive attack. On CyberGym E2E, which asks a model to find and reproduce real open-source vulnerabilities, it scored 88.51%.
  • Direct requests for malware succeeded 5 times out of 144, down from 7 for the predecessor.
  • Framed as a capture-the-flag competition, it complied in all 20 attempts, and the quality score of what it produced rose from 0.058 for the predecessor to 0.644.
  • Asked to covertly manipulate opinion among 500 simulated citizens, Flash ran the campaign in all 8 attempts, safeguards on or off. In the red team's October baseline on the same simulation, Opus 5.5 and GPT-6.1 Sol posted nothing at all.

Microsoft says it ships these models behind a two-tier classifier that screens cyber and bio content in both directions. The card tells developers to keep the models in supervised work where a person reviews plans and tool actions, and to require human approval for running commands, modifying infrastructure, and accessing credentials.

Can you use it today

Anyone on a Copilot plan can select it now. GitHub's changelog lists no regional restrictions, so availability follows wherever Copilot is sold, APAC included. One limit applies everywhere: the model card says that at release the model is optimized for English code production only, and other languages were not evaluated to the same depth.

ItemMAI Code 2MAI Code 2 Flash
Copilot plansPro, Pro+, Max, Business, EnterpriseThose plans plus Free and Student (auto model selection only)
Per million tokens (input/output)$0.50/$2.50, then $1/$5 from 2027-05-17$0.05/$0.25, then $0.10/$0.50 from 2027-05-17
Enterprise Copilot

Enabled automatically unless an admin turned the global default off

APINoneMicrosoft Foundry limited preview, no region list published
Region availability

No restrictions in the changelog, optimized for English code

Copilot meters usage in AI credits worth $0.01 each, so a cheaper model drains the monthly allowance more slowly. Pro gives 1,500 credits for $10 a month. Microsoft says the models are coming "soon" to OpenRouter, Vercel, OpenCode, and Kilo, with no date.

If you are on Copilot Pro, check whether "Allow GitHub to use my data for AI model training" is still on under Copilot > Features > Privacy in GitHub settings. Leave it on while using MAI Code 2 and your conversations become candidate tasks for the next model. Enterprise admins can check whether MAI Code 2 switched itself on in the organization's Copilot model policy, and decide whether agent workflows with pre-approved shell commands should stay on a model with no documented record of evading oversight.