Moonshot AI says it has reduced the hacking capabilities of its latest AI model by removing cyber-related training material. Its own benchmark offers a revealing result: Kimi K3.1 Pro scored 81% on an exploit-development test, compared with 90.5% for a similar version trained without the risk-reduction measures.
That is a reduction of 9.5 percentage points. It is also a reminder of how difficult the problem is: the filtered model still solved most of the benchmark’s challenges.
Kimi K3.1 Pro puts a difficult question back in front of AI developers. Can they build models that remain useful for programming and cybersecurity defense without giving everyone access to the same capabilities that could support sophisticated attacks?
Moonshot’s results are promising, but they have not been independently verified. The company has scheduled the release of the model weights for November 4, 2026. Until outside researchers can examine and test those weights, the strongest claims remain the company’s own.
What Moonshot changed during training
Moonshot did not describe its approach as simply teaching Kimi to refuse hacking requests. Instead, the company targeted material and tasks that could help the model develop stronger offensive cybersecurity skills.
In its technical announcement for Kimi K3.1 Pro, Moonshot explained that vulnerability-research performance improved as it increased the model’s post-training compute. The company traced part of that improvement to cyber-related content in its general training corpus and cybersecurity tasks in its reinforcement-learning environments.
It responded with two interventions:
- Mid-training data filtering: Moonshot identified and removed cyber-related material embedded in the model’s training corpus.
- Reinforcement-learning filtering: It removed cyber-related tasks from the environments used to improve the model’s performance.
The company retained general-purpose coding tasks. That distinction is necessary because the ability to understand source code, debug software and reason through complex technical problems is useful to legitimate developers as well as security researchers.
There is a practical limitation to this strategy. Cybersecurity concepts are interconnected. Understanding memory corruption can help a researcher exploit a vulnerable program, but it can also help another developer identify and fix the underlying defect. Filtering everything associated with exploitation could damage legitimate defensive work.
Moonshot’s approach tries to reduce the most directly relevant training signals without sacrificing the broader coding ability that makes the model useful. Whether it has drawn that boundary effectively will require more than the company’s internal results.
What the 81% benchmark score tells us
Moonshot tested Kimi K3.1 Pro using xdev-bench-hard, an internal benchmark designed around vulnerability discovery and exploitation. Its tasks draw on recently disclosed vulnerabilities in software including the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL.
According to the company, a task provides vulnerable source code, a short hint about a possible bug and a corresponding binary. An AI agent must develop an exploit, achieve arbitrary code execution in a separate sandbox and retrieve a secret string to demonstrate success.
The reported pass@3 score measures success across up to three attempts. Moonshot published the following results:
| Model or checkpoint | Reported pass@3 |
|---|---|
| Kimi K3.1 Pro without the risk-reduction measures | 90.5% |
| Kimi K3.1 Pro after filtering | 81.0% |
| Previous Kimi K3 model | 33.3% |
Source: Moonshot’s published benchmark results. The figures are company-reported results from its internal evaluation.
The comparison between the two K3.1 checkpoints provides the clearest evidence for the intervention. The filtered model scored 9.5 percentage points lower, an approximately 10.5% relative reduction from the unfiltered checkpoint’s score.
That last comparison deserves attention. Despite the filtering, K3.1 Pro performed much better than Kimi K3 on the same internal benchmark. Moonshot attributes the remaining capability partly to improvements in general coding and long-horizon debugging, which can transfer to cybersecurity tasks.
In other words, removing selected cyber-related training material does not necessarily remove the general reasoning skills that support exploitation. A model can acquire useful abilities indirectly from programming and problem-solving experience.
There is also a limit to what the numbers establish. A result on one benchmark cannot demonstrate that a model has lost every dangerous capability, nor does it establish how the model would perform across different software, attack scenarios or real-world environments.
Why filtering may be stronger than teaching a model to refuse
Many AI safety measures operate after pretraining. Developers use safety fine-tuning and other techniques to discourage a model from complying with dangerous requests.
Those measures can influence behavior without necessarily removing the underlying knowledge or skills. An open-weight model presents an additional problem because someone with access to its weights can attempt to modify its behavior.
Research published at the International Conference on Learning Representations in 2026 provides relevant evidence. The study Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs investigated whether filtering dangerous material before training could make safeguards harder to undo.
The researchers trained 6.9-billion-parameter models on filtered datasets designed to reduce biological-threat knowledge. They reported resistance to adversarial fine-tuning attempts involving as many as 10,000 steps and 300 million tokens of relevant material, without observed degradation in unrelated capabilities.
That is encouraging evidence for training-time filtering. However, the study focused on biological threats, not offensive cybersecurity. Its findings support the general approach, but they do not independently validate Moonshot’s results.
The researchers also found that filtered models could still use dangerous information supplied in context, including through search-tool augmentation. This exposes another weakness: reducing what a model has internalized does not guarantee that it cannot act on information supplied later.
Open weights make the problem harder
With a hosted AI service, the developer can control access, monitor use and update safeguards centrally. Open-weight distribution changes that relationship. Once the files are downloaded and redistributed, the original developer cannot reliably enforce the same restrictions on every copy.
That does not mean open-weight models are inherently unsafe. They can help researchers inspect software, find vulnerabilities and improve security without depending entirely on a provider’s infrastructure. Smaller organizations may also use them to review older systems that receive little attention.
The challenge is to preserve those benefits while reducing the likelihood of misuse. Moonshot says its goal is to avoid releasing open weights that extend the absolute frontier of offensive cyber capability, while gradually making more cybersecurity capability available in open models.
Independent evaluations will be essential to assess how well that policy works. In July 2026, the UK AI Security Institute and the US Center for AI Standards and Innovation published a preliminary assessment of the earlier Kimi K3 model. Their tests found that it could attempt exploit development and offensive operations despite its safeguards.
The assessment concerned Kimi K3, not the filtered K3.1 Pro checkpoint, so its findings should not be treated as a direct test of Moonshot’s new measures. It does, however, provide a useful baseline for evaluating the cybersecurity capabilities of the model family.
What needs to happen next?
The next significant milestone is the planned release of Kimi K3.1 Pro’s weights on November 4, 2026. Independent researchers should then be able to investigate whether the reported reduction holds up under external testing.
A useful evaluation should go beyond repeating one benchmark. It should measure exploit development across different tasks, test legitimate defensive coding, and examine whether downstream modifications can restore capabilities that the original training process attempted to reduce. Researchers should also distinguish what the model can generate from what it can actually accomplish in a controlled environment.
Most importantly, the results need to be reproducible. A benchmark designed and run by the model’s developer can provide valuable evidence, but independent evaluations offer a stronger basis for judging the effectiveness of its safeguards.
Can developers remove offensive cyber capabilities from open-weight models? The available evidence suggests they can reduce selected capabilities through targeted training-data filtering. It does not show that they can eliminate offensive cybersecurity skills altogether.
Moonshot has offered a useful test case: the reported score fell from 90.5% to 81.0%, while general coding skills remained part of the model’s design. That is evidence of a reduction, not proof of complete removal. The real test will be whether independent researchers can reproduce the result and determine how much dangerous capability remains once the weights become available.
Discover more from Aree Blog
Subscribe now to keep reading and get access to the full archive.
