AI Regulation: How Open Source Is Being Instrumentalized

Open source has always carried a normative weight in technology policy. It signals transparency, community, and democratic access. In the context of AI regulation, these associations are being leveraged by nearly every actor in the debate, often toward contradictory ends. Governments invoke open source as a path to accountability. Companies invoke it as a reason they should be exempt from regulation. Researchers invoke it as a prerequisite for scientific reproducibility. Civil society warns that the term is being diluted to the point of meaninglessness.

The reality, as a ProMarket analysis from the Stigler Center at the University of Chicago documented, is that "open source is having a moment in AI regulation," but what that moment actually produces depends entirely on how we define the term and what obligations we attach to it.

The EU AI Act and Open Source: An Uneasy Exemption

The EU AI Act, which entered its enforcement phase in 2025, includes specific provisions for open-source AI models. General-purpose AI (GPAI) models released under open-source licenses benefit from reduced compliance obligations. They are largely exempt from the transparency and documentation requirements that apply to closed commercial models, provided they do not pose "systemic risks."

The logic behind this exemption is understandable. Requiring a solo researcher who publishes model weights on Hugging Face to produce the same compliance documentation as OpenAI or Google would be disproportionate and would chill open research. The regulation attempts to calibrate obligations to organizational capacity and commercial intent.

However, the exemption creates a strategic incentive. As PromptFoo's regulatory overview notes, large companies can release model weights under an open-source label and thereby reduce their compliance burden, even though they retain exclusive control over the training data, the infrastructure, and the institutional knowledge required to reproduce or meaningfully audit the model. The exemption was designed for independent researchers but can be exploited by well-resourced corporations.

The Case for Open Source in Regulation

The strongest arguments for open source in AI regulation hold real weight.

Auditability. When model weights are public, independent researchers can probe for biases, failure modes, and safety issues without depending on the model provider's cooperation. The entire field of mechanistic interpretability depends on weight access. Closed models can only be evaluated through their API, which limits analysis to input-output behavior (including multimodal capabilities) and excludes internal representation analysis.

Reproducibility. Science requires that results can be independently verified. If a company claims their model achieves a certain benchmark score, the claim is only verifiable if outside parties can run the model under controlled conditions. Open weights enable this.

No single point of control. When a model is widely distributed, no single entity can unilaterally revoke access, modify behavior, or impose usage restrictions after the fact. This is a meaningful safeguard against both corporate and governmental overreach.

Community oversight. Open models benefit from a distributed security model analogous to Linus's Law in open-source software: "given enough eyeballs, all bugs are shallow." The more people who can inspect a model, the more likely subtle problems are to be identified.

The problem is not with these arguments but with how the term "open source" is being stretched to cover practices that deliver none of these benefits.

Open-Washing: The Gap Between the Label and the Practice

The term "open-washing" has gained traction to describe the practice of releasing model weights while withholding everything else that would be needed for genuine transparency. A truly open AI system would include the model weights, the training data (or a detailed description of it), the training code, the evaluation methodology, and the safety testing results. In practice, most "open" releases include only the weights and inference code.

Consider the spectrum:

• Meta's Llama series: Released under the Llama Community License, which permits most commercial uses but includes revenue thresholds and explicit restrictions on using the model to train competing models. Meta does not release training data, detailed training code, or the full evaluation infrastructure. This is "open weights" but not open source in any traditional sense.

• GLM-5 from Zhipu AI: Released under Apache 2.0, a genuinely open license with no usage restrictions. However, the model was trained on proprietary Chinese-language datasets that are not publicly available, and the Ascend-specific training infrastructure is not reproducible by most organizations. This pattern is increasingly common among Chinese AI labs: the weights are open, but the process that produced them is not.

• DeepSeek V3: MIT licensed, with a technical report that describes the architecture and training methodology in unusual detail. Still, the training data is proprietary, and the specific infrastructure optimizations that enabled the sub-$6M training cost are not fully documented.

• OpenClaw: An autonomous agent framework that releases its full codebase, but this openness created immediate security consequences when the code was deployed in production without adequate hardening.

The ProMarket research found that when policymakers reference "open source AI," they are typically imagining the full-stack transparency of traditional open-source software. What they are getting, in most cases, is weight release by companies that retain exclusive control over every other part of the stack. This gap between expectation and reality is where instrumentalization occurs.

The Concentration Paradox

The most fundamental issue is one of economic structure. Training a frontier model requires tens of millions of dollars in compute, access to massive curated datasets, and a team of specialized researchers. In 2026, fewer than fifteen organizations worldwide have demonstrated the ability to train a model that competes on major benchmarks. Releasing weights after training does nothing to change this concentration.

This creates a paradox. Open weight release is framed as democratization, but the power to create the thing being democratized remains concentrated in a small number of hands. The community can fine-tune, quantize, and deploy, but it cannot meaningfully challenge the architectural and data curation decisions that determine the model's fundamental capabilities and limitations.

This is not an argument against weight release. Open weights are strictly better than closed weights for the research community and for downstream users. But framing weight release as sufficient for transparency, and using it to justify regulatory exemptions, conflates access with accountability.

Standards and Benchmarks: Who Decides?

A related concern is evaluation governance. When we say a model is "safe" or "aligned," we are making a claim that depends on specific benchmarks, specific evaluation methodologies, and specific definitions of harm. These standards are currently set by the same organizations that train the models.

The open-source community has made progress here. Efforts like EleutherAI's Language Model Evaluation Harness, the Open LLM Leaderboard, and various community-driven red-teaming initiatives provide independent evaluation infrastructure. But these efforts are chronically underfunded relative to their importance, and they struggle to keep pace with the rapid model release cadence.

The EU AI Act recognizes the need for harmonized standards but largely delegates their development to existing standardization bodies (CEN, CENELEC) whose experience with AI systems is limited. There is a real risk that the standards that emerge will be either too vague to enforce or too rigid to accommodate the pace of technical change.

What Europe Should Push For

If European policymakers want meaningful AI transparency, several concrete steps would help.

Define "open" with precision. The term "open-source AI" should come with clear criteria analogous to the Open Source Initiative's definition for software. Weight release alone should not qualify for regulatory exemptions. A tiered system, where increasing levels of openness correspond to increasing levels of compliance relief, would better align incentives.

Fund independent evaluation infrastructure. Public funding for independent model evaluation, red-teaming, and safety testing would reduce the dependency on model providers to self-report. The EU's AI Office could establish or fund an independent body specifically for this purpose.

Require training data documentation. Even if training data cannot be fully released (due to copyright, privacy, or other legitimate concerns), detailed documentation of data sources, curation methodology, and known limitations should be a baseline requirement for any model deployed commercially in the EU.

Support compute access for public interest research. If concentration of compute is the bottleneck for meaningful openness, public investment in research compute (along the lines of the EuroHPC initiative, but targeted at AI training) would lower the barrier to independent replication and verification.

Engage with the global dimension. Chinese labs are now releasing frontier models under permissive licenses. The regulatory framework needs to account for models trained under different jurisdictions with different norms around data collection, content moderation, and government access.

Beyond the Label

The instrumentalization of open source in AI regulation is not a conspiracy. It is the predictable result of a term with strong normative associations being applied in a domain where its traditional meaning does not straightforwardly transfer. Source code and model weights are different kinds of artifacts. The transparency that open-source software provides, where any competent developer can read, understand, and modify the code, does not automatically follow from releasing a 400GB blob of floating-point numbers.

This does not mean open source is irrelevant to AI governance. It means that the policy conversation needs to move beyond the binary of "open vs. closed" toward a more granular understanding of what specific forms of openness actually deliver in terms of accountability, safety, and public benefit. Getting the definitions right matters.

Sources: ProMarket / Stigler Center, "Open Source Is Having a Moment in AI Regulation: Here Is What the Data Says" (2025). PromptFoo, overview of AI regulation frameworks and open-source provisions. EU AI Act text, Articles 52-53 on GPAI model obligations and open-source exemptions.