I‘m not even sure what the argument for banning Chinese models/open weights even is supposed to be?
1. if it’s to stop hackers doing hacking things with „uncontrollable models“ then, well… they’re already doing something illegal to begin with, why would they care about breaking another law running these models?
2. if it’s to stop foreign actors, then that ban would not apply to them anyway
3. it’s not stopping distillation either, Chinese labs are already banned from using US frontier models and look at how good that is working
I don’t get it. Am I missing something? The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term. But thats also an admittance that the American labs can’t compete on merit anymore, and should by itself also limit the viability of the idea that all those VC billions will ever make a return? In any case this would be something benefitting only a very few for a short time (labs + investors).
Someone please enlighten me what the actual argument here is, cause I can’t see it.
> Someone please enlighten me what the actual argument here is, cause I can’t see it.
> The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term. But thats also an admittance that the American labs can’t compete on merit anymore
That's the argument. To be precise the publicly stated argument is that they're attacking American providers by distilling. The real aim is to eliminate competition because otherwise Anthropic and OpenAI are non viable and the US views them as crucial for winning the 'AI race' which they see as putting whoever wins it on top in terms of warfare/economic power etc.
But isn‘t the whole argument of „winner takes all“ already dead, if china is such a big threat that the US labs need import controls?
Distilling or not, they are clearly close enough to the frontier, that the supposed „free market“ country needs market controls. That may work for US markets. But not the rest of the world.
Idea: petrodollar policy becomes „tokendollar“, enforced by US military dominance. If you force me to pay altman at gunpoint, then maybe ill stop using kimi
They are simply out of ideas to keep the most expensive party in the history of mankind going.
That's it. Simple as that. When you grasp for straws in panic mode, you don't exactly spend time strategizing and weighing the pros and cons of each straw carefully.
Decade? We're 4 years in since ChatGPT released publicly and have gone from "Wow this thing is stupid" to running coding agents on auto mode, and the Chinese labs have been around for even less time. The US providers may lose dominance in 1 or 2 years.
In some portions they already lost dominance. My claude will happily generate disgustingly vulnerable code then flag me for ToS violation when I ask it to make sure the code that it just made, in the same context window, in the same conversation is safe and doesn't have glaring holes. Also great to be charged $xx.xx+ amounts and then it refuses to answer without refunding it.
They won't. China doesn't have the training capacity and the quality data sets, the former because of access to fabs, the latter because of internal censorship that is getting worse by day.
The current panic is happening precisely because Chinese open weight models like Kimi and Qwen are demonstrably competitive with SotA models like Fable, but far cheaper.
No, the Chinese tech companies routinely super optimize for a given 'famous' or 'established' or 'mainstream' benchmarks. Any other concern is secondary.
In other words, they look good on surface, but suck whenever anyone put them to any serious use. That's why they are cheap, they have to be cheap because they suck.
Having limited amounts of high quality Chinese data to train on doesn’t really decrease or affect the appeal of these models to the rest of the world. I don’t think the CCP is restricting or limiting the datasets that models can be trained on.
Also the models themselves don’t even seem to be censored or restricted that much. If the Chinese government is fine with the guardrails being on the API level and maybe even restricts the usage of external providers that’s again a net benefit to everyone else since they would have little incentives to force Chinese companies to lobotomize their models.
You are probably about capacity so we can only hope that Huawei and others can catch up and break Nvidia’s monopoly (since Intel and AMD suck too much too much to accomplish anything useful that’s the next best thing)
Also Chinese labs are actually still publishing research publicly which alone would accelerate and increase the competitiveness of open AI models developed in the US or even Europe.
> doesn’t really decrease or affect the appeal of these models to the rest of the world
Anyone who tried using Chinese models for serious programming tasks ditch them quickly if they have a choice.
> I don’t think the CCP is restricting or limiting the datasets that models can be trained on.
What I meant was censorship limits the amount of high quality training sets by limiting the amount of all training sets from which high quality sets grow out from, e.g. the entire set of posts on Baidu forums before 2017 is gone forever.
> so we can only hope that Huawei
Chinese companies do not have access to commercial-scale advanced nodes. The impact is at best negligible.
> Anyone who tried using Chinese models for serious programming
Perhaps, but that’s because they are just objectively worse for many tasks than GPT/Claude. I don’t see how that ties to censorship inside of China.
> censorship limits the amount of high quality training sets by limiting the amount of all training sets
Well again, I don’t really understand how is the lack of high quality Chinese datasets matter much if they have access to English/etc. datasets. That’s only an issue to their users inside of China.
> not have access to commercial-scale advanced nodes
Well the gap was way bigger a few years ago. If Chinese companies can at some point produce GPUs that are competitive cost wise i.e. they are willing to tolerate significantly lower margins than Nvidia (whose margins are obscene) that’s not a huge issue.
> I don’t see how that ties to censorship inside of China.
LLMs are parrots, when they are trained on English data they behave like English / western speakers which carries the __western__ values.
China would never want a shiny model deployed by some solemn state-own institute to pitch western values like democracy or freedom of speech.
So China is better off starting with Chinese content in the first place, but only to find out it shoots itself in the foot, because censorship __removes__ Chinese content from the Internet, and it __prevents__ truth-reflecting, i.e. quality, content from appearing.
When this is realized to be not sufficient, and they have to use English content, the English content needs to be censored, too.
So compared to other models trained on the uncensored set, the Chinese models are trained on the data sets that is likely smaller. And since high quality content is suppressed, there is likely less high quality training sets. Therefore, their models likely suck.
(Meta knows the importance of high quality training sets so that it started distilling its own employees who are smart.)
> produce GPUs that are competitive
The most recently known advanced process in China is some kind that requires multiple exposures in DUV machines. This already limits the cost to a minimum that's probably not economical.
For GPUs to be competitive, there's more to the hardware. Driver also matters. GPU drivers these days are compilers in disguise. How many high quality compiler projects have we heard of from China?
It's unlikely for both the hardware and the software of a Chinese GPU to deliver what's boasted in its marketing material.
> carries the __western__ values…China would never want a shiny model
Maybe, but that’s an opinion on what you think might happen. So far that has not been the case.
But yeah currently you can ask their models exactly the same controversial question (from the CCP perspective) in Chinese and the answer would be very different than the one in English.
> And since high quality content is suppressed, there is likely less high quality training sets
For now it seems that the Chinese labs are more interested in maximizing the quality and competitiveness of their models instead of lobotomizing them to enforce compliance with some sort of political agenda.
That might change of Chinese models might surpass Western ones in the future. Which is why competition is necessary to discourage them from doing thay.
> It's unlikely for both the hardware and the software of a Chinese GPU to deliver what's boasted in its marketing material.
Not currently sure. Chinese cars were pretty crap as well a few decades ago (and same applies to countless other example) so there is at least sufficient precedent to believe that China might close this gap eventually (and they clearly have been over the recent years when it comes to GPUs)
well free markets meant something very different in adam smith's time; what they meant by free was 'free from rents' whereas today its usually mean in the milton freedman terms as 'free from any interference' which is how i interpret the parent poster...
This Schroedinger's Freedom is getting tired now. They happily label themself as free-market democratic liberal progressive when you're justifying bombing a poor third-world country, but suddenly claim to be unfree undemocratic oligarchic whenever anyone asks them to act on that principle.
Yes, it’s freedom both from state interference but also from monopolies, oligopolies and trusts using their position to suppress the competitiveness of other market participants. Which is generally impossible to prevent without the state interfering in the markets.
Which is why a free market requires regulation, but only for the purpose to break up trusts, cartels and monopolies, and to ensure equal access to the market and a level playing field.
It's not a trivial problem to solve and probably requires constant readjustment, and there's probably plenty of room for ambiguity in assessing which decision would make the market more free or whether something is or isn't hurting market freedom. But some regulation is undeniably necessary.
I would include consumer rights protection and regulating uneven and unbalanced contracts. To an extent one might expect that sufficient competition could solve this but often it doesn’t. You usually are forced to buy goods/services as a bundle and don’t get to pick the terms and conditions.
While there might be other companies offering better terms the product quality of the first company might outweigh that from the perspective of the consumer. Of course one might argue that if a company is in a position to do that it’s effectively a monopoly and if all major market participants are engaging in that it’s effectively an oligopoly (or a case of implicit collusion)
And another (maybe less controversial example) is establishing minimal quality standards for at least certain products like food and drugs (i.e. the laissez faire offhands approach to that e.g. in the early 1900s resulted in some severe issues) because effectively that means that companies are selling products with hidden defects.
How could you tell how close they are to the frontier if they weren't distilling? Clearly those labs think distilling is worth it despite the large effort they need to go through to bypass anti distillation techniques used by the frontier labs
Anthropic is “distilling chinese models”. Everyone uses outputs from every other frontier model in both pre training and post training, hilarious to give it this name and pretend like it’s an attack. Fable is just a distillation of stolen copyrighted works, etc.
The argument is - these models costs hundreds of billions to develop - to BOTH US and China. China is giving them for free despite not making financial sense because they want to weaken US's tech firms ability to compete - because China sees AI dominance as critical for their national security.
The US also sees AI dominance as critical for their national security, and relies on a market economy and non-state-owned labs. This means that these frontier labs are truly susceptible to "predatory pricing" (economic term for a competitor selling at a loss to eliminate you), which is illegal in the US and any other free market economy exactly because of its implications on market efficiency.
I hate it just as much as the next guy, but the fact the discussion here ignores the fact that these concerns and dynamics are real just lower the discussion level instead of actually discussing potential solutions.
With that said, what I'm worried about is that the government solution will be far more hostile than some semi-ban on open models. An example of an even worse scenario, they could take over frontier labs and restrict access to everyone in the public (which might still not solve espionage).
> China is giving them for free despite not making financial sense because they want to weaken US's tech firms ability to compete - because China sees AI dominance as critical for their national security.
Nobody is making this argument because this is a normal thing that companies do. It's so common it is named in business strategy: "commoditize your complements."
There's a 2002 Joel On Software post explaining this using tech examples that were already old in 2002[1].
In the present, Meta is doing exactly the same thing with Ollama as Alibaba (who is funding some of the Chinese labs). Alibaba needs advanced models internally, and does not want to have a dependence on US models that can be arbitrarily shut off by Washington. Alibaba also wants to drive usage of its cloud among companies who are now demanding competitive AI models (which Alibaba may be locked out of providing). However, selling AI is not its core business. Putting cheap AI in the world creates demand for cloud services, which is part of Alibaba's core business.
Not only does this make financial sense for Alibaba, it is fairly basic.
Google gives away consumer software like Chrome and Android (not their core business) to drive search, which is their core business.
et cetera. The frontier labs are exposed in exactly the same has been way every pure play tech company before them. The US labs chose quality as their moat; remains to be seen whether that was a good choice, or if they can add another moat in time.
> Putting cheap AI in the world creates demand for cloud services
No, Alibaba doesn't have the right amount of hardware to serve these models.
The part that benefits Alibaba in these open models is to maintain its public reputation so as to maintain its stock price. The core that drives everything else inside Alibaba is its e-commerce.
> Nobody is making this argument because this is a normal thing that companies do. It's so common it is named in business strategy: "commoditize your complements."
It is common in business strategy, far less so in nation-state governance.
The reason you, me, and so many people in the United States still have a far better quality of life and level of freedom than most people in China is because it’s bad to run a country like a business, full stop. When you do so, you kill far more people.
> It is common in business strategy, far less so in nation-state governance.
It is not clear why this is bad for Alibaba or China, could you expand on that?
> it’s bad to run a country like a business
The "Chinese miracle" resulting from China's integration with the global economy has lifted ~800 million people out of poverty and raised standards of living quite rapidly overall. A lot has been written on the topic, but suffice to say that it's a fairly unorthodox argument to make that the Chinese have not been well-served by the arrangement.
> level of freedom
The context of this discussion is whether the US government should prevent US citizens from accessing yet another product that is freely available in the rest of the world, to benefit a tiny handful of rich people in California.
Let me be clearer: if you do not hold stock in OpenAI or Anthropic, you likely stand to benefit from more AIs being provided by more providers. Imagine if the government had banned MySQL and PostgreSQL, so we all had to buy databases from Oracle and Microsoft. That would be strictly worse for everybody except employees of Oracle and Microsoft. This is exactly the same thing.
For private companies to an extent yes. In public companies that’s just a way to increase your employees compensation without harming your free cashflow. I don’t think tax wise it’s any different than them paying you more in cash and then you buying the same amount of shares.
> The reason you, me, and so many people in the United States still have a far better quality of life and level of freedom than most people in China
I think it's because every single time the farmers in south america or the philippines try to organise and get out of poverty you guys either bomb them or arm some terrorist to kill them.
But sure tell yourself otherwise if that makes you feel better.
> you, me, and so many people in the United States still have a far better quality of life and level of freedom than most people in China
Whether this is true or not is still open to debate, but the fact that it’s presented as some kind of absolute truth that needs no substantiation is simply ridiculous.
Well no.. regardless of anything else the median American generally has q higher standards of livin than the resident of pretty much any major country (including Europe). While compared to e.g. Germany or France those in the lower percentiles are probably doing worse China specifically still has a huge amount of poverty. Even if you happen to be legally allowed to live in one of the major cities and have a decent job.. well 966 is still a thing and you don’t get paid $500k like the OpenAI employees allegedly forced to sleep under their desks.
> the median American generally has q higher standards of livin than the resident of pretty much any major country (including Europe)
If you say so. I visit the U.S. quite often, and the homeless and mentally ill people on the streets there don’t seem particularly happy. I haven’t seen anything like this on such a scale either in the country where I was born, in the countries where I used to live, or in the country where I live now. But yes, the index suggest otherwise, so I suppose it’s just my misconception, sure.
Yes, that’s exactly what I said. The median and above are doing very well in relative terms, the bottom line 10 percent or so are often doing much worse compared to other developed countries.
That’s extremely American centered, though? Despite the constant complaining the median American is still very well off compared to almost everyone else. If you go to the top 10 percentile or so them maybe only Switzerland and some very small countries (tax havens, petro states and such) can really compete.
> The US also sees AI dominance as critical for their national security, and relies on a market economy and non-state-owned labs.
[...]
> but the fact the discussion here ignores the fact that these concerns and dynamics are real just lower the discussion level instead of actually discussing potential solutions.
We had this problem solved for DECADES. Build the best post-secondary education system in the world. Lure all of China's (and the rest of the world's) best and brightest. Educate them. Give them well deserved, high paying jobs doing what they love and invite them to build a life and stay here.
Took me a while but for the west the value of education isn't primarily in jobs but in democracy. It puts the bar much higher than we've ever accomplished. People are supposed to run th county (believe it or not)
I think the misconception here is that they "give them away".
Sure, you can download the weights and run it yourself, but to run a 1T+ parameters you need roughly 2TB of VRAM (for full precision). That costs around $500k to $1M, not including electricity and rent.
In reality, they provide their models priced via API or subscription just like any other company. The "free weights" are only useful for other research labs. It seems that China understands that research should be shared and open, which is one of the reason that AI has been able to develop that fast in the last 10 years.
> free weights" are only useful for other research labs
No, they are useful for any compute provider who can host these models and sell inference. Which effectively puts a cap on whatever the Chinese companies can charge. So longterm that means that their margins are limited to cost + some low margin.
If they are making money by selling tokens then anyone who has sufficient compute to host their models can make more than the Chinese labs since they don’t have any R&D costs.
I mean I think the primary reason is really the attention is all you need paper and meta's "leaking" of meta's llama.
China is plenty happy to steal IP and just say "this is just how business is done in China it's a cultural thing" but quickly run to defend their own IP in western courts when stolen by western businesses. It works one way.
And I have experienced this firsthand when having something manufactured there. Exact clones appearing on Alibaba just days after.
> costs hundreds of billions to develop - to BOTH US and China
OpenAI and Anthropic are extremely bloated compared to the Chinese labs. They have thousands of extremely well paid people working for them. Deepseek I think has a couple of hundred and I doubt they are paid a lot.
Of course compute for training is expensive but I doubt that’s necessarily the overwhelming majority of the operating costs for Anthropic/OpenAI
> on a market economy and non-state-owned labs
Sure the private companies developing these models are heavily interlinked with and even maybe partially owned by the government. But its a matter of degree. Since that’s the situation in the US similar in how these companies both shape government policy directly and in how the government is constantly intervening and shaping their behavior outside of any regulatory framework.
The US government is both heavily protectionist and interventionist these days.
> "predatory pricing" (economic term for a competitor selling at a loss to eliminate you), which is illegal in the US
It is very funny that this was/is the playbook of a huge number of successful, VC-funded, "disruption" plays.
Cheap cabs that undercut the market, wait for taxi companies to die, raise your rates now that your "rideshare" app got everyone hooked on the subsidized rate, IPO!
The software landscape is full of free products that are used to upsell other services. In this case the upsell is inference at data centers. China has built a big advantage in Green energy and is working at the GPU issue both on the hardware and model level. Those are the two main cost inputs into AI and should allow China to compete. The exact model being used is not going to matter as much as the cost that companies must pay.
> This means that these frontier labs are truly susceptible to "predatory pricing" (economic term for a competitor selling at a loss to eliminate you)
OpenAI and Anthropic were losing money too until recently, when they started focusing more on increasing revenue (which is also part of why companies are now looking more at Chinese models to reduce costs).
That's nonsense, people pretend that a "frontier model" is the most important technology ever, but we know for sure it cannot be because several companies are doing copycats of one another. It is not China that is doing much effort to copy-cat, it's the US companies that are copying one another. The only real barrier in this technology is how many Nvidia chips you can buy and how much data you can get to train your LLM. All these models are basically moving in the same trajectory, it is essentially a very expensive copy-paste job they're doing.
US didn't have that much rational reasoning when it killed Japanese exports in cars, chips, and finally software. That in turn lead to China simply taking the seats Japan used to occupy, but as far as imposition of these protectionist means is considered, US does not need that much sophisticated justification internally.
I find such beauty in the fact that the robbers are getting robbed but it's all part of progress overall. It's not that China won't come up with what the US are coming up with. Just a matter of when it would happen.
I am sure that some other country (Mistral from France, India, Canada etc.) would have even more chances to innovate perhaps.
and because Chinese models are open-weights, the distillations effects of them could be easier done as well ;)
In my opinion, its a win-win plus even within worst case scenario*, I already believe that the current open weights models are in general speaking good enough perhaps for my and other use cases as well and I feel like we will probably most likely get more open-weights model for a long time in general as well perhaps.
The other countries don't have cheap capital, cheap infra, or scale. The talent is not the bottleneck. Look at how many SV companies are helmed by Indian executives. If the Europeans cared an iota about AI progress, Huggingface wouldn't have been headquartered in NYC.
Yep. We'll steal China's IP as well so all of this really is just no big deal. Everyone is a robber in some way.
Though one must ask, why are American models banned in China? Hmmm.
> It's not that China won't come up with what the US are coming up with. Just a matter of when it would happen.
I agree. All this discussion about China releasing open weight models is amusing. It's like ok they release open weight models.... and so what? If they leapfrog ahead of the US then we'll just undercut them with our own open weight models.
> Though one must ask, why are American models banned in China? Hmmm.
Because the models are behind a gate and the data goes overseas. The same is not true for open weight models. One should avoid the former and prefer the latter simply based on principle, not on country of origin.
> Though one must ask, why are American models banned in China? Hmmm.
Because they're not? But the American government and Anthropic don't want American models to be used by the Chinese. Anthropic is notorious for its effort to tell if the user is within China.
China is taking the moral high ground when Anthropic bans users from China. But if Anthropic didn't do it, the GFW would have surely banned Anthropic because Claude will say lots of things that do not agree with the official narrative.
That's just conjecture though surely. Like not saying you're wrong but the same thing would result from crawling and using results from Chinese Web for training along with everything else. If the most prevalent model people are posting responses from online in Chinese is qwen then of course that would happen. There's no evidence to say either way really.
wow if they're that crucial, but economically nonviable, we should probably just nationalize them and fund the superproject with taxes, then put the results into the public domain!
> But thats also an admittance that the American labs can’t compete on merit anymore
This is exactly it.
Try to buy a BYD in the United States. You can't (without a complicated process) because they're so much better cars than our domestic brands that our domestic brands couldn't compete and lobbied to keep them out.
> should by itself also limit the viability of the idea that all those VC billions will ever make a return?
It just has to last until the next quarter / fundraising round.
I can understand (not necessarily defend) protectionism for products with nonzero manufacturing cost and some required physical size and such.
I am befuddled when the same policies are applied to products with marginally zero manufacturing cost, no real physical size, and such. Regulating ideas is hard folks.
>I am befuddled when the same policies are applied to products with marginally zero manufacturing cost, no real physical size, and such.
Because it's not about perfect obstructionism, it's about adding red tape to businesses that end up relenting and going to a frontier model. If this gets these frontier models an extra few million dollars in exchange for greasing some palms, that's a business success.
The first token on a new AI rig costs $X (full capex cost) then every token after that costs virtually zero. Over time the cost/token trends towards zero (modulo opex). That said, AI does have higher opex than general SaaS so it can’t get as close to zero.
But that’s kind of a different question, the running of some service. The “product”, the model, is a collection of files. The “manufacturing” required to add another customer is “send them the files” and has ~zero marginal cost.
There is a finite number of tokens that a rig will turn out over it's lifetime. Divide the cost of the rig plus electricity by that number and you have your cost per token. Yes providers can screw up on scheduling and end up paying more than they should, but that's not magic.
The AI rigs cost money, sure. But no one is talking about regulating those. We are talking about regulating models, which do have zero cost of reproduction.
It’s like banning the leaked DeCSS key. Good luck.
If you mean being forced to overpay for no-longer-SOTA closed models, that’s I guess the protectionist motive here: impossible to enforce, enforcement uneven and political, illegality rampant, I suppose all by design too. I’d say it’s brilliant if it weren’t so evil. I have hope saner minds will prevail.
objectively true. And there's zero need to factor in things like "human rights" or "environmental protection", because these are costs that have been externalized and the consumer don't pay for it (in their lifetime anyway). That's what objectively means.
Chinese cars are unavailable in the US because US manufacturers are 100% unable to compete. Japan was in the exact same position before, but because of ally status, japanese car companies (with their gov't as the proxy) managed to negotiate and have a "voluntary import restriction" in place to protect US manufacturers. This artificial restriction created a higher price for consumers than it ordinarily would have been.
The japanese car manufacturers were also able to setup shop in the US, and instead of exporting, they manufacturered directly on US soil. They're allowed to do this sole because they're an ally, and i don't expect this same to be allowed for chinese companies.
If you consider Tesla in China to be Chinese, then BYD in EU would be Hungarian in the parent comment. Japanese brands also manufactures in Mexico, so Mazda is suddenly Mexican now? I've never seen anyone claiming such stuff, except when it comes to anything China.
Model Y remains American just like iPhone, regardless of where they are made. Nobody refers to iPhone as Chinese or Indian.
But what are you trying to argue here? "America doesn't abuse it's workers as much so it's more ethical to buy american EVs that don't exist!"? We're objectively behind because so many EV lines were paused/cancelled last year
>Read the news in the past few years?
Yeah, I recommend doing that. We're going to have encyclopedia's dedicated to what's happening in the US in these last few years(and coming next few years).
>after factoring in harms they do to human rights and the environment
America c. 2025 and beyond has no leg to stand on in terms of espousing care of human rights and environment.
>even EU despite claiming to be morally superior to the US, heavily tariffs BYD (much more than the tariffs that the globe mocks Trump for).
You are technically correct... because Trump did not actually Tariff BYD directly. The 100% EV tariffs was a Biden era move, made to give American manufacturers time to develop their own EV solutions for the domestic market. Separate from whatever global tariffs Trump announced.
And then trump ended EV credits, and ultimately pushed the road down to American manufacturers giving up on their EV initiatives.
You're not seeing it because you're thinking in terms of internal US economics, but this is a question about international economics and geopolitical leverage in the distant future.
The reason to consider the ban is because it might be the only way to preserve a fully autonomous and independent American frontier AI stack and the long-term strategic value of possessing such a stack could vastly outweigh the cost of giving up true free market competition on AI. If giving US startups and other companies access to cheaper Chinese AI means sacrificing the US's ability to own its own frontier AI stack, is that a rational trade, or would it severely and irrecoverably sacrifice the country's technological autonomy and leverage for decades to come in exchange for cheaper tokens for a little bit early on?
If the US not only gives up most of its manufacturing capability to China, but also allows itself to give up its own AI stack and become almost entirely dependent on foreign AI, then it's conceivable the combination of the two sacrifices will deal a permanent deathblow to the country in exchange for what will turn out to have been a couple decades of cheap goods and AI tokens.
It puts all US companies at a price disadvantage and forces American company to shoulder the load of training frontier models most will never need while the rest of the world has cheap AI access.
The keyword in your sentence is "access". The rest of the world would have cheap access to AI, but no ownership of anything at the frontier. If the US joined the rest of the world, declined to develop frontier AI, and allowed itself to become dependent on cheap Chinese AI, once that dependency was fully realized it would eventually translate into a lack of access to the frontier. That doesn't matter until, suddenly, it becomes the single most important thing to the entire country's future.
China is making a simple bet: that the US will offshore AI in favor of cheap tokens, just like the US previously offshored manufacturing in favor of cheap goods. They're doing this because they know that if they alone possess frontier manufacturing and AI capabilities, then they alone can build the world's most powerful technologies in the future which combine the two (e.g., robotics that will revolutionize all their industries, domestic life, and military far beyond any other country).
> The keyword in your sentence is "access". The rest of the world would have cheap access to AI, but no ownership of anything at the frontier.
So... just use distillation and create their own frontier models?
I don't see the problem here.
If the goal is to hold sovereign ownership over your own SOTA models, China has already shown that any nation could achieve that pretty easily if they need to.
And that's ignoring the fact that many of these models are open weight so hosting/owning sovereign inference/intelligence itself is entirely a hardware problem.
This whole conversation reminds me a lot of North Korea deciding they needed their own OS when Linux is right there. If the technology is open and available to all, then sovereignty over this core technology is irrelevant.
Linux is not a relevant comparison, because the fundamental capabilities of an OS do not grow exponentially with time or investment.
If building full sovereign ownership of the production chain for frontier-level AI were so easy, there wouldn't only be two countries on the entire planet that have that right now. If distillation worked in the way you seem to imagine it does, nearly every country in the world would have a full sovereign frontier AI stack.
They would if they cared to invest in it. That they've not is a policy and economic choice (possibly because they don't have the same sense of panicked urgency I see among a lot of folks around here), not evidence of some insurmountable barrier to entry.
You also jumped to a conclusion that I don't think is self-evident: that owning the production chain matters.
Assuming open weight models continue to advance, who cares? Just let the US and China expend the compute on model development.
And if they close up, then fine, do what China did and build up a domestic industry and distill as a way to get a jumpstart. We already know that's possible since China already did it.
It's well understood at this point that China's results are simply not purely the result of distillation. They're the result of significant investment with a willingness not to make a profit any time soon. That's why distillation is not enough. Again, if it were that easy, then it would in fact be that easy. And all the evidence across the world clearly illustrates it's not. Even with distillation, it's incredibly expensive building a fully sovereign domestic frontier AI stack.
> Assuming open weight models continue to advance, who cares?
There's no reason to assume they will advance as fast as closed models. If the world is reduced to a single fully autonomous producer of true frontier intelligence (China), and everyone else is using open models produced from or distilled from the results of China, then two categories of AI inevitably emerge: open models and frontier closed models, the latter of which are known and controlled only by their sole producer. Given that progress is exponential, once competition is eliminated, one can reasonably expect an exponential gap in technology capability to emerge between China and the rest of the world. That is not a side effect but the entire point. It's the very goal being engineered right now.
> And if they close up, then fine, do what China did and build up a domestic industry and distill as a way to get a jumpstart.
"build up a domestic industry" will be about as easy as reshoring frontier manufacturing capability. That is to say, there's a threshold of "no return" after which it won't be possible at all without extremely painful re-architecting of the entire country's economic system to force a massive domestic development that said system no longer naturally incentivizes.
>because the fundamental capabilities of an OS do not grow exponentially with time or investment.
I strongly disagree. Especially in times where Microsoft is actively trying to ingrate OS into Windows itself. the OS clearly isn't just a schema for apps and a task scheduler anymore.
>If building full sovereign ownership of the production chain for frontier-level AI were so easy, there wouldn't only be two countries on the entire planet that have that right now.
The rest of the world is still cooling from COVID, so not exactly a great time for any other country to spend a trillion dollars on speculative technology.
Regardless, this is the wrong lens. The two countries aren't avoiding it because it's hard, they avoid it because it gives less power and money to them.
You can disagree, but simply stating disagreement isn't substantive. Linux can be enormously important while also not being subject to exponential value creation through improvements to itself.
I gave an example. And I was thinking of genuine features and responsibilities for a competitive OS, not "value creation" (I don't base my OS usage on how much shareholder value it generates). The capabilities of an OS in 1996 and 2026 are massively different.
You gave an example of Linux having value, which is not contested and not relevant. You did not give any examples showing the existence of an exponential here. Moreover, the value being discussed is long-term geopolitical leverage resulting from a country's sovereign technology capability that remains unperturbed under heavily adversarial conditions, not shareholder value. In fact, it is a direct corollary of my claims that the two concepts of value are not equivalent, and that is precisely the trap being set and the gap being exploited.
>long-term geopolitical leverage resulting from a country's sovereign technology capability
Do I really need to explain in this community how Linux went from some underground freeware to the most used OS in the world? How is that not "sovereign technology capability".
If a machine uses software in any way shape or form, it's about 70% likely Linux powers it. Doesn't matter what industry or topic it is. America only remained a superpower this long because it was an early adopter of modern compute and shaped much of its GDP around software powering every industry.
The only thing not sovereign about it is that many technologies are technically holed up in private companies, that the US government contracts. But I don't know how you can say the growth hasn't been exponential.
I feel we're nitpicking over minutae at this point so I'll cut it off here.
You are free to bow out. However, we are decidedly not nitpicking over minutiae. I don't think you have understood this thread, and that's leading you to make very confident assertions that are individually correct but just not relevant.
There's a kernel of a good idea in what you just wrote: open infrastructure can be part of a nation's sovereign technology stack. That's true, and Linux is indeed an example of it. But that has little to do with anything I claimed.
Linux is fundamentally not relevant because investing more in it does not confer exponentially increasing geopolitical leverage to a single nation uniquely above all others. Next-generation frontier AI, however, appears to have that property: if possessed by one nation, that sovereign production capability appears so far to confer exponentially increasing international leverage.
I think it would be helpful if you explained how distillation works (as relevant to the conversations threads here) because I agree that this is a key point.
Spot-on and really well written post highlighting the challenges here. Really it just comes down to the US and surrounding sycophants and China. There just isn't any other game in town. For the US to just, well, give up on AI development would be civilizational suicide. I wouldn't put it past us though, most Americans in particular are so weak minded they only care about cheapest possible product. It's interesting to see how well propaganda campaigns have been working to harm American interest in AI and related technology (data centers are a bit of a different story). Young students are applauding guest speakers giving them the news they want to hear about how AI is bad and they need to tear it all down, while in the real world those same students are actively committing career suicide by refusing to innovate or adapt. It wouldn't really matter except that they vote, and those votes will wind up doing harm under the guise of doing good, but really it's a matter of delusion that we will all pay the price for.
Anyway the good news is everyone says China's strategy of just releasing open-weight models is the coup de grace for American technology companies and all the investment, research, all of that stuff by all of these companies who certainly don't care to survive and continue making trillions of dollars and can't possibly be managed by the same genius engineers who post on HN will just go away.
And then once that happens we'll just do what China did, and let them build a bunch of AI capabilities, scale out data centers, invest trillions, then we'll just release open-weight models too and then what?
You don't need to know a whole lot about AI to realize that most people on the Internet that are commenting on these stories haven't thought about how the real world works for more than two seconds when it comes to this stuff. China can enact a strategy, we can copy that strategy and even improve upon it.
Surprise Pikachu.
You (not you) can't claim a lead is unimportant and this undercutting strategy is so good and then simultaneously say the US can't do it back to China. That's dumb. If the lead doesn't matter everyone will stop doing AI research because being undercut is too expensive. How likely does that seem, and why isn't China stopping research if that is true and they just need to keep releasing open-weight models? Sorry, but being on the leading edge matters a whole fucking lot. As I've written in other posts this is like a cheap Android phone from Wal-Mart versus the latest iPhone. There's a reason you buy the iPhone and not the cheapest Android phone you can find even though they have all the same apps and both make phone calls.
I'm open to changing my mind here, but I have yet to see a convincing argument. Just a lot of pearl-clutching and China fear mongering.
We can see the same situation play out with manufacturing capability.
The USA has recognised that it needs more manufacturing capability. But rather than just build that capability (which is what the Chinese state-planned economy would do, and did) it tries to incentivise private companies to build it. Which didn't work because the companies building domestic manufacturing face competition from Chinese imports. The investment money can get better returns chasing property deals or rent-seeking monopolies.
In the "we can do it back" case, it still takes a US private company a lot of time and money to distill their model off the Chinese model. There has to be a lot of profit for them to do that. That profit isn't really available - it's a competition with all the other models, and no monopoly rents. There's for sure a viable business there, but it's a "lifestyle business" not the 1000x growth business that VC's want.
> We can see the same situation play out with manufacturing capability.
I don't think manufacturing is a great analogy and we should be careful to try and compare these two very closely. But the closest comparison would be iPhone versus Android. Many more people use Android, and the phones are a lot cheaper and can be produced in mass, but the iPhone is the dominant phone and captures much more market share.
> There has to be a lot of profit for them to do that.
Sure... but if China is just subsidizing AI companies to copy/distill/whatever American models it's fair for the US (as it would be for manufacturing) to work to block those models. But also you have to remember that the US is in the lead, so while there isn't profit that I'm aware of to be had now, if the Chinese caught up then presumably there would be some level of profit. There's good and bad here though too because, well, as you can see in China's over-built housing market the government makes a lot of bad assumptions and mistakes and is subject to forces that can cause less than desirable outcomes. It goes both ways a bit.
Lots of people want this outcome. We now know that the closed-weight model isn't an effective moat, but if the closed-weight model business fails, the open-weight business has nothing to distill, and model progress slows.
I suspect this is part of why China dipped into their vast strategic oil reserves to reduce purchasing and offset the shortage caused by Trump's blunder in Iran. If energy prices get too high, training will slow. They know they can pirate our models at 95% fidelity, so they want training to continue so they can pirate the next ones also.
They've invested in capabilities that undercut attempts at AI supremacy by US companies. Every time they put those capabilities to use:
- They look more and more like somebody who can prevent whatever world domination plans appear to be brewing in the minds of US billionaires. They can turn this perception into leverage over other countries by threatening to close their model weights.
- They get to use the improved models to boost their economy's efficacy (they probably think they can put it to use better than anybody else, and they might be right).
- For every $100 that the US paid for a given unit of improvement, they just pay $1. The longer the US keeps this going, the less stable it will become because the sacrifices the US has made to pull it off are causing things like measles and explosive diarrhea and will ultimately lead to unrest. Meanwhile the longer China keeps this going the less likely the US is to stand in their way, because the US will have neglected overlong the things that make it a threat to China (the legitimacy of the dollar, soft power programs like USAID, a thriving economy).
Also they have leverage over US because they've recently demonstrated that they can raise the price of oil. That means that they can pop the AI bubble anytime they want. So the longer the US is dumping everything into model improvement, the more opportunities they'll have to use that leverage.
> The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term.
> Someone please enlighten me what the actual argument here is, cause I can’t see it.
But you did see it.
The US has invested trillions in AI that the companies involved are never going to make back. Even without competition from open models, but definitely not with it. And if those trillions turn out to be worthless, that's going to have a massive impact on the market and cause a lot of bankruptcies.
Banning those open weight models isn't going to fix everything, but it would the impossible obstacle slightly smaller.
> The US has invested trillions in AI that the companies involved are never going to make back. Even without competition from open models, but definitely not with it. And if those trillions turn out to be worthless, that's going to have a massive impact on the market and cause a lot of bankruptcies.
This is the point. And what nobody seems to bring up that inference is a massively profitable margin. The labs can still grow at 80% margin as opposed to 99% margin (I do not have the numbers). But that will hurt the valuation and investors.
With protection from Govt, you can compete at 1.5 or 2x of the equivalent open weights price. But it is not sustainable to keep the prices high, just a matter of time.
> The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term
It won't even do this. Streisand effect will probably draw even more attention to the open weight models
The entire marketing narrative in AI already operates this way --- "GPT 2.0 is too dangerous to release, oh noooo!" etc
After a report from RSA Security, who were in a licensing dispute with regard to the use of the RSA algorithm in PGP, the United States Customs Service started a criminal investigation of Zimmermann, for allegedly violating the Arms Export Control Act.[5] The United States Government had long regarded cryptographic software as a munition, and thus subject to arms trafficking export controls. At that time, PGP was considered to be impermissible ("high-strength") for export from the United States. The maximum strength allowed for legal export has since been raised and now allows PGP to be exported. The investigation lasted three years, but was finally dropped without filing charges after MIT Press published the source code of PGP.[6]
I remember t-shirts with the DVD encryption key on them when the authorities tried to ban that. This shit always backfires. You'd think they'd know this by now.
The CFO or wtv that tweeted, their argument was, if you allow cheap free competitive models, you will stop the influx of capital in frontier development of even more powerful LLMs, thus capping how far the tech could reach. Obviously, they're mostly asking for regulation to save their money, but it seems their argument is that without an environment that rewards the development of the next frontier model the tech might not evolve as fast.
> if you allow cheap free competitive models, you will stop the influx of capital in frontier development of even more powerful LLMs, thus capping how far the tech could reach
Hmm. So either the weights are open for all to use, or nobody trains those weights at all? Sounds like the ideal outcome to me.
The worst possible outcome is the one where frontier labs train godlike AIs then kick the ladder out from under them. That should be prevented at all costs. No country or corporation can ever be allowed to win the AI race, for technofeudalism will follow swiftly after. As long as they keep competing with each other, we're the winners. If that music ever stops, it's time to watch out.
I'd rather have AIs achieve sentience and wipe out humanity as a whole than live in a world where they submit to the whims of gigacorporations and totalitarian governments for the sole purpose of efficiently oppressing me and keeping me in line while they enjoy riches and freedom I'll never have.
> This is precisely what the people worried about AI risk and job displacement want, so they should be strongly in favor of open weights then, right?
I'm getting a "gotcha" vibe from how you worded the question, but the answer -unironically - is "Yes." Anyone who is not a closed-weight AIaaS provider[0] will have their lot significantly improved by equally capable, open-weight models.
0. Or their investors. I'm surprised to see Y Combinator signed this letter sering their holding in OpenAI is worth billions and hasn't IPO'd yet. I suspect they crunched some numbers first before signing.
Edit: I realized another group that may be unhappy with frontier open-weight models are those who believe LLMs are inchoate super-intelligences; not only do I disagree with them, but if they are correct, I don't see why we should trust trillion-dollar corporations and their out-of-touch CEOs with that responsibility, when countless CEOs have shown time and time again, that they are not aligned with humanity's interests.
There are two subsets of that group who tend to oppose open weights:
1) The people who think LLMs are "inchoate super-intelligences", but then your argument is entirely correct and they should be happier to see less exuberant development of the things than to make sure the rapid advancement of the world-destroying technology is locked up inside corporate entities that are themselves poorly-aligned. So these people either don't actually exist or are making an honest mistake and should update.
2) Disingenuous sock puppets who pretend to be the first group because they want regulatory capture and know that "we need to regulate this because it's unsafe" sells more tickets than "we need to regulate this to exclude competitors and drive up margins".
I just don’t buy it. There’s still gonna be demand for stronger models. Sure growth will be slower, but this might even drive a push for more cost efficient training / inference and/or new architectures if money is harder to come by.
Yeah, I agree with this take. When there's less capital floating around it will make these labs prioritize shrinking the models so less hardware (or older hardware) is needed to run a model with the same or similar capabilities. They'll have to do that simply because the cost of running the models will become more important.
It also might end up pushing us towards LLM ASICs assuming the model progress slows significantly.
Which is kind of weird, imo. Shouldn't increased competition spur increased investment? If your competitor's product is approaching or beating yours on some metric, isn't the answer to improve your product to differentiate it from the competition? To say that open weight models will stop the flow of capital seems to indicate one of two things: 1) the frontier labs have no idea how to compete, so they are giving up; or 2) they figure that investors will eventually shrug and say "eh, the open weight stuff is good enough, no need to do more research," and retreat from the industry altogether. In the former case, it makes sense to lobby for regulation because the alternative is failure. In the latter, it's an admission that their product has no differentiating qualities from any other in the category and the product is more like a commodity or a utility and not a machine to print infinite wealth.
How about: excellent technology will demand premium prices, and hence will have the capital to back it ... horseshit technology will die a death, and any horseshit capital that backed it.
Opening up will make things more efficient and at least give said horseshit capital a chance at being redeployed elsewhere a slightly higher chance of being successful.
VCs have been horseshit for some time and throwing money at horseshit startups to increase their AUM, their incentive structures are fundamentally misaligned with their clients.
AI labs and investors are scared, and pushing administration to ban them, because they can't compete with Chinese models soon, similar to how they banned Huawei, and Chinese cars.
When there is cheaper alternative, companies might go with self hosting option, which reduces the enterprise moat of AI labs
I read a book and learned something, did I "borrow" or "distill"?
I read info from a website without their consent, then learned something, and now I am teaching that knowledge to others, did I "borrow", "steal", "distill" or doing piracy be selling it?
How would it even be stopped if you could ban it? It's not like you're banning just one company. Anyone with the resources can still obtain and run the models. Sure, big models like Kimi K3 will be less accessible until a foreign version of Openrouter with no incentive to listen to US law begins offering it. Or they offer it and obfuscate what model it actually is behind codenames.
It wouldn't surprise me if that kind of problem is partially why weights are open. Because you can't meaningfully stomp them out completely.
I don't think there actually is a congnizant argument other than to protect investments. And of course they cant say that.
You are using way to much logic in this. You are witnessing powerful people who are operating way outside the scope of their own abilities thrash around.
AI isn't being stopped. Definitely not by people who value money. The inertia at this point is probably even too much for those who have some sort of ideological framework against it.
To those of us who value ideas and progress, especially a non human-centric idea of what progress could look like, the path looks pretty clear.
there is no stopping this. We are working everyday in so many small ways, not to capture value, but to advance the machine.
the idea that some decree from an ageing narcissist and a pyramid of lackey bureaucrats is at all meaningful is just hilarious. Of course it's meaningful to the "economics". But many of us could really not care a lick as we don't do it for money.
It's hilarious because at this point it's clear they can say that. We've just had a coordinated talking points push that voter picture IDs should be required because "Olive Garden".
Well, that's them doing the same thing, yeah? They can't say that they want voter picture IDs for the real reason (partisan electoral strategy), so they have to constantly litigate electoral fraud and the whole parade.
Working so far keeping Chinese EVs out of the U.S.: U.S. car company CEOs say publicly that allowing them in will destroy the domestic auto industry overnight.
I agree that many of the rich and powerful are not at all as ingenious, creative or capable as they like to make themselves out to be, but I don't see a clear/easy way to escape the economic ramifications of the fact that they own and control virtually all physical resources and all political power. If, say, Chinese develop AGI, and that makes US tech companies obsolete - I see them banning Chinese AGI, even going to war, rather than ceding power (realistically there may be some backroom deal), they would grind the population into dust if it let them retain their wealth and power.
This. Living/Coming from a country like that, when shit gets real the elite will squeeze every inch regardless of how brutal it gets. It happened to the USSR and every other country. The US is not that different.
Would it not be more logical that US billionaires simply switch sides and move to China? I am not convinced that Americans are as patriotic as they claim and willing to go down with the ship.
In some sense that's happening. A Reichsfluchtsteuer of some kind is happening where wealthy people from Western countries have to pay exit taxes. Americans can't exit without giving citizenship, though.
That’s what I don’t get tho, I think in today’s world they totally could?
„We need to protect US investments as our entire economy depends on this market sector“ seems like something that would be soothing to me as an investor?
It’s not like the big money givers can’t see that china is close, it’s been the news multiple times already. But taking active policy steps to protect against that should boost market confidence, no?
Instead we get alibi arguments to accomplish the same thing in the end?
I think you are confusing the tiny amount of people who are actually investors or stand to directly benefit from these companies with the overall population - the overall population being generally against AI at this point.
'We must protect VC profits at the cost of the rest of the economy and our entire 'free market' system' probably doesn't sound as good.
No I’m not confusing them. I am very aware it’s mainly policy for a minority of people. But this minority drives the markets.
This administration has done a lot for that minority already so I don’t think anyone would be too surprised? In this case they can even claim to protect the greater economy, a crash of which does impact the majority.
„US good, China bad“ has been a long term talking point by Trump too and this would neatly fit into that to be consumed by his base
Nobody would be surprised, and something like that is what's bound to happen to some extent. But I think the current tech/VC cronyism and corruption is becoming more and more evident to the masses, and opposition is growing. There's also the economic factor of US tech being dominant in the world, if US decides to ban better tech because it is too competitive, then US companies cannot really compete globally anymore - they can't outwardly signal that they are being outcompeted.
Sounds familiar, I used to think the same about the Internet and free software. Turned out, the old world digested the unstoppable march of progress just fine.
The only possible legit argument for blocking Chinese models is to force US orgs to use US based providers/models. And despite the fact that I and a lot of others cannot or should not use Chinese based models, they should be generally available as much as anyone in any other nation has access.
It's a lot like the guards against dealing with security issues.. I'd rather they be found and fixed then to try to stop people from being able to discover in the first place. I mean, where does that stop, do we let AI security scanners not tell you you have a SQL injection for fear someone might exploit it? They already can and they might use something else that tells them as much.
The argument is that it will make $OAI and $ATHR billions of dollars because U.S. customers with deep pockets will have nowhere to go but the "vetted as safe and verified model providers" aka the duopoly. No one with two brain cells to rub together takes anything they or the current U.S. government has to say at face value.
This is a tale as old as the industrial revolution. I can only hope people see the trick for what it is this time.
> I don’t get it. Am I missing something? The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term.
You say you don't get it then you say exactly why they are doing it. You get it just fine.
Actual arguments are only there to make govt do something. Intent is to stop competition. With Chinese models being let's say 6 months to year behind it's quickly shrinking profit margin that even if we hit the pie in the sky AI usage numbers the American companies want us to believe, they wouldn't be able to live off with that competition
I don't think we can separate out an enlightened argument from the total politics of out time. We are amidst a struggle between labor and capital, nationalism, and the commodification of humanity's most impressive engineering feat. There is a subset of humans who are romanticizing foregone times with artificial intellectualism. This group as significant control over media and are very good at oersuadion through emotions rather than reason.
Your argument about not successfully competing doesn't hold because distillation is essentially model stealing. The situation is like trying to compete on effort in school on an exam against someone who's copying your answers. Of course the person copying your answer will always have to study fewer hours.
I sympathize with the argument saying that they ripped the whole Internet and books first though of course.
More broadly, this is one of the things I dislike about the US political system. When executives or major shareholders can legally make very large political donations while their companies are lobbying on issues that directly affect their businesses, it creates an unavoidable perception problem.
There is no coherent reasoning here for banning open weights. The billion of dollars to train a model was supposed to be a moat so they can sell tokens. AI labs can't make money by capturing values from intelligence if any company could get the weights to build their own AI token factory.
I wonder how they would feasibly ban it. They could do so with some infrastructure offering to run it, but ultimately you would be able to run your own private instance. What am I missing here, how would the ban be implemented?
It’s to try and direct investment capital into US foundational model firms and to capture that market globally. It requires the containment of China, and cutting it off from the rest of the world, which is the broader geopolitical play at work
Same as the tariffs, more or less. Both hinge on the same argument: foreign competition will drive down profits for domestic providers, so banning the foreign competition (or making them more expensive) will benefit domestic players
I've said before that I think red team security researchers should be able to investigate the security of government and corporate systems without needing permission.
Currently companies say "you cannot investigate the security of our systems without our permission, because it is our system and we are responsible--also, if there's a data breach, we aren't responsible." See how that works? As a society we seem fine with this.
We are sacrificing national security for the convenience of companies.
So... to answer your question:
Probably, again, we will sacrifice national security, and even our national competitiveness in this important new AI industry--we will sacrifice it all for the convenience of companies and so they can make a few bucks.
It's more important than ever that the good guys actively monitor government and corporate systems for security flaws. These open-weight models are going to be in the hands of the bad guys; the hackers are coming whether the company is ready or not. Ideally a good hacker will find and disclose the vulnerabilities before a bad hacker pwns half the nation's personal data for the 3rd time this month--or worse.
This will require that we stop bullying security researchers. Things like the State Governor personally pushing for felony charges because somebody pressed "view source" on an HTML page shouldn't happen.
They will not actually shut it down. Rather, they will simply continually posture in that direction and release a statement that the wrong lawyer can interpret the wrong way. This will make it a soft no-no for anyone involved in government procurement which is most big companies thus ensuring american AI companies have the market, while also ensuring startups, etc still have access to the structurally lower-cost chinese models (otherwise, they will simply resort to the black market which is why they will never enforce a full ban). In most US-adjacent countries like India, anyone that sells to US companies I see avoiding chinese model providers for the same reason. The exception is fine tuning and running it on their own hardware, which naturally everyone is completely fine with since no money flows to the chinese model company.
Tl;dr, it's purely an economic argument. Every major argument in the world is about money and thus power, don't bother analysing it technically.
> In most US-adjacent countries like India, anyone that sells to US companies I see avoiding chinese model providers for the same reason
But the argument isn’t about model providers tho right (providing inference)? Using Chinese providers is already a privacy problem if user data is involved, so I don’t know if anything would change here.
What you’re suggesting is that the policy would stop big co‘s from running their own inference completely, no? Or only the foreign open source models?
> don't bother analysing it technically
I feel you have to, because where’s the line? Is running GPT-OSS myself fine (if I were a big company)? Mistral models? Or are none of them ok? What about if I am getting inference from a small startup, which is running OSS models for me? Or is only OAI/Anthropic ok?
Nothing will be truly banned. As in, you can always run and use all the chinese models. Startups will use them. Like I said, they will simply use soft regulations to ensure that all the major players who due to power law will form a majority of the economy, will be using OAI/Ant. If JP Morgan wants to run their own Deepseek V4 on their own hardware, why would the US government care? They simply don't want them using chinese model providers. That is all. However, today, the fact of the matter is that frontier model self-hosting in Fortune companies is DOA, everyone is using APIs. I am not saying this can never change in the future, but seeing past trends, people don't even host their own frikkin source control, so I find it hard to believe JPMC is going to host Deepseek V7 or whatever. You might, and I might, but only the top few thousand mega enterprise deals matter economically speaking. Doordash using qwen for dish classification is completely irrelevant in the grand scheme of things (which is not to say it won't be profitable for alibaba).
It is not clear that "model self-hosting in Fortune companies is DOA".
Among the chiefs of the Fortune companies there is no complete unity, as some think that they should receive some of the AI profits, instead of OpenAI and Anthropic.
Recently, both the CEO of Palantir and then the CEO of Microsoft have criticized quite harshly the business model of OpenAI and Anthropic, implying that it is bad both for society and for their corporate customers.
None of those two would come first in mind for altruistic behavior, so their criticisms must be based on business reasons.
For the CEO of Palantir, it is clear that the criticism was a sales pitch for a new Palantir product (Sovereign AI), which is a turn-key system including all hardware and software needed by a company to self-host AI model inference and also training or fine tuning.
It is less clear which is the motivation of Microsoft. If they do not like the business model of OpenAI and Anthropic, then this means that they also want to compete with them, but that cannot be the same as what Palantir proposes, i.e. self-hosting on premises. Perhaps Microsoft also wants to propose some kind of self-hosting, but instead of on premises, doing it on some kind of Azure instances.
So the Chinese are not the only "enemies" for OpenAI and Anthropic.
Oh yeah I've been seeing the nadella posts on twitter, which I presume are aimed at those reporting to him primarily. Agreed on them and palantir. But even if that is the case I believe the risk due the "soft regulations" I mentioned above and the lucrative customers being government or major government contractors would mean them using openai/anthropic models maybe on a separate on-prem deployment or existing government azure deployments. My statement as phrased was too strong I responded to a picture in my head of JPMC deploying open source models using their own tech team.
> But thats also an admittance that the American labs can’t compete on merit anymore
Merit was never competitive. Mass consumption habits select hard for cost, and this is an industry driven entirely by scaling factors. If you can scale for cheaper, you win. Ivy League-decorated researchers aren't doing RnD for free. The frontier labs can only go so cheap before they're so far in the red that VC gets cold feet. Leaving the expensive frontier research up to private enterprise wasn't a long-term strategy.
> I don’t get it. Am I missing something? The only thing a ban would do is protect the American market from further downward price pressure on inference
What's not to get? Do you understand how much money is tied up in the industry? Do you know what happens if that goes up in flames? In the middle of a massive asymmetric economic depression? Why on earth do you think they would just sit there and watch it happen? Because of political virtue signaling about free markets? You know, I have a bridge to sell you if you're interested.
> Do you know what happens if that goes up in flames
I think in capitalism thats called a market correction?
> Do you understand how much money is tied up in the industry?
If the free market „virtue signaling“ no longer matters, then they could’ve also put a stop-gap on investment volume to protect the economy against over-reliance?
Or how about go full plan-economy and just dictate price per token until investors get their money back?
I’m joking around but of course I understand whats tied up in this. I just don’t see how banning open source models is helping any of that in the long run.
> then they could’ve also put a stop-gap on investment volume to protect the economy against over-reliance?
Which instantly collapses the load bearing bull market. They can't do that for the same reason they can't do anything about the insanity of the real estate market. The fallout would be existential. The economy isn't rational, remember. It's highly reactive and, no matter how strong your state is, always out of your control at the end of the day.
America is years away from being able to rotate meaningful volumes of capital into heavy industry. Until then, AI has to outpace the cooling of the tech sector, while also contributing to it. The entire political strategy in play is a frantic spinning plate game. There is no long run if it fails, so that is their strategy as far as we can see. Or maybe they're just maliciously incompetent. Given the American administration opaquely decided starting a forever war in the Red Sea was a solid choice at this point in time, possibly to try and fluff up an auxilliary bull market with the MIC, we really can't rule that one out.
> Or how about go full plan-economy and just dictate price per token until investors get their money back?
I've heard people talk about AI as being like a utility, like electricity. My understanding is this is how electric utilities work. So it's not as impossible as you might think.
Similar market correction happened in 2008 and the the US has definitely not gone more free-market-let-the-bad-businesses-fail direction since then. Rich a-holes always find ways to build systemic risk where they can reap the rewards and socialize the losses.
> I‘m not even sure what the argument for banning Chinese models/open weights even is supposed to be?
Easy: you're a traitor, if you use foreign AI for the fraction of the cost. It's unamerican not to fill the American billionaires pockets. Free markets exist, as long as you buy America first!
This is less so a hedge against Chinese labs catching up to the frontier/OAI Anthropic falling in valuation, and more a hedge against a complete recession. We are overleveraged in AI investment and if tomorrow any Chinese Lab can drop a model at the frontier what was the point of all those billions. Now, if you can protect the American model from the bubble popping even for just long enough to hedge risk a little better, then you may just prevent collapse. The problem is though, that the market is heavily irrational right now and if anything this may just be the signal to investors to double down on American AI.
Suppose a foreign actor deliberately builds an unaligned model, and dumps it at a very low cost through subsidies. Since the cost is lower, everyone integrates it into their business. But now the foreign actor has control; perhaps the model checks for instructions to execute, perhaps it has backdoors, perhaps... who knows?
I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".
The problem with this argument is that sharing model weights only has downsides for said foreign actor compared to providing a cloud service. Since the suspicion exists, security companies are going to comb over the weights and discover every hidden secret of the model, while with a cloud provider you send your most sensitive data to a remote server, and trust the AI lab wont train on your data, with the only shield being a TOS. While with a local modal, even sending a peep about your internal data without cause would be scandalous.
It's not even going to be a good honeypot - since said actor doesn't really provide inference, people will have to pay for and run the infrastructure of these models, which bad guys cannot even subsidize, unlike cloud based provides.
This is literally what all AI is right now. Anthropic has been paying companies to use its product to form that dependency and maybe they're not misaligned but there's no regulation in the US that'd in anyway deter them from purposefully misaligning their model.
Sorry, to clarify - this isn't just "offering it at a loss" Anthropic is directly paying PE firms to enroll their held companies in their services[1]. When it comes to a question of whether this is a national security attack vector I think there isn't a cut and dry argument and people could go either way but... private companies are not necessarily aligned with the US government even when contracted with the US government (see Starlink Ukraine shenanigans) and the steps the US government usually would take with highly critical infrastructure to ensure alignment haven't been taken - in my opinion at least, this is definitely a very subjective area.
Beyond certain obvious 'third-rails' even experts have reasonable disagreements on what 'aligned' means.
> everyone integrates it into their business
Few large corporations would sole-source integrate a closed model developed in a country designated as a foreign adversary (which could cut off access at any time) or which their own country might restrict.
> now the foreign actor has control
Your concern is one reason why the Chinese are making most of their models open weight.
> it's not as trivial as "regulatory capture".
You're right, it's not just regulatory capture but the vested interests trying to achieve regulatory capture are certainly leveraging concerns like yours to deceptively achieve that capture. In the case of an open weight, domestically-hosted model, the threat vector you're concerned about isn't a threat in the same way as a Chinese-made, closed source data center router or 5G phone switch.
I am not supposing a low-risk event. This is an obvious attack vector that an adversary would almost certainly exploit. I would exploit it if I were them.
You're supposing an event that there's no evidence for happening and that is on par with 'let's ban open source software because some open source software could be malicious'.
Let's not forget that as of now most models are like game cartridges i.e. they need a 'host' to run i.e. llama.cpp - they're not executable code by themselves.
I could see an argument for a security review prior to integrating a model into a highly sensitive industry/agency, but that is standard for any software.
It's certainly not an argument for banning open models because US AI corps don't want to compete with them.
Imagine a model that's 10000x more intelligent than any human. You integrate that model into a sensitive industry. You now effectively have an extremely powerful foreign agent running e.g. your energy sector. This is qualitatively very different than a backdoor in Redis.
I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.
Seems a bit far fetched, given that I’ve never seen any PoC/research on this topic. Got any pointers?
To be fair the fix to this seems to be that American labs would need to adjust to the new „fair market price“ then, right? Subsidy-backed competition is still legitimate competition in capitalism (e.g. Uber).
And you could still ban specific, known malicious models/labs instead of blanket open-source bans. What about Mistral models, for example.
It is much more difficult to embed a "sleeper agent" in an LLM than it is to do the same in any closed-source program. Any proprietary program, e.g. from Microsoft, is much more likely to contain hard-to-detect "sleeper agents", than any LLM.
An LLM is a collection of inert data, it is not an automaton.
You obtain an automaton only using an inference program, like llama.cpp or anyone of the many others, and then using some harness around the inference program.
You can put in the harness all kinds of guards, to detect any possible harmful action and prevent it.
In something like a CPU from Intel or AMD it is easy to put some undetectable backdoor that would allow the remote control of the computer, unless you disconnect any antennas and you filter all wired network traffic through an external firewall that would allow only whitelisted connections.
On the other hand an LLM is a much less plausible attack vector, as long as you host the inference yourself. Only an LLM that runs externally, e.g. at OpenAI or Anthropic, can be really dangerous.
I'm not advocating for the policy, I'm just saying that there is a legitimate argument that's not so easy to dismiss.
> And you could still ban specific, known malicious models/labs instead of blanket open-source bans
With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)
> With models 10000x more powerful, we would need to be extremely conservative
I agree, I even think we’re already there if yesterdays OpenAI/HuggingFace story is legit.
We need to start building these agent systems in a way so that it won’t matter if a model is compromised. Malicious models is one way, but prompt injection is also still an unsolved problem.
And if you solve that, then a malicious model is also no longer a threat. Data exfiltration maybe, but IMO US labs are probably also doing that for training, even if saying otherwise (of course I have no evidence of that, but the temptation must be insane)
It's to give unfair advantage to Anthropic and OpenAI who would then be expected to shower Donald Trump personally with hundreds of billions of dollars in cash, airplanes and tacky golden things.
The problem isn't so much investors losing money, it's if this kills the economy.
A lot of elites are now unemployed with nothing better to do thanks to AI. Historically speaking, populist uprising are easily squashed except when you have a group of counter-elites supporting the movement. We're finally starting to see the results of that with the Deflock movement cutting down cameras.
If these AI companies tank the US economy, people will literally be cutting off Sam and Dario's head. For good reason.
Don’t even need to look into Trump’s pockets. The entire US economy and the bet on AI depend on OpenAI & co. doing well. If you can download a Chinese model and run it on your own hardware instead of giving money to an American corp, a large part of their valuation vanishes into thin air, and so does the rest of the US economy that desperately needs AI to do well.
I admire China for its cunning, having put the US on this spot in their long-going economic war. If unchecked, it’s in China’s interest to release to the world for free models that can compete if not surpass the frontier ones.
> But thats also an admittance that the American labs can’t compete on merit anymore
It’s more an admittance that the capitalist model can lose to a more socialist approach, unless you “fight fire with fire” and use the government to distort the market for you.
1. if it’s to stop hackers doing hacking things with „uncontrollable models“ then, well… they’re already doing something illegal to begin with, why would they care about breaking another law running these models?
2. if it’s to stop foreign actors, then that ban would not apply to them anyway
3. it’s not stopping distillation either, Chinese labs are already banned from using US frontier models and look at how good that is working
I don’t get it. Am I missing something? The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term. But thats also an admittance that the American labs can’t compete on merit anymore, and should by itself also limit the viability of the idea that all those VC billions will ever make a return? In any case this would be something benefitting only a very few for a short time (labs + investors).
Someone please enlighten me what the actual argument here is, cause I can’t see it.