Anybody could release an open model tomorrow. Google is the only US based lab releasing open weights models. OpenAI released one once, which might or might not count as "releasing", depending on your definition
You forgot the GPT-2 that came long before that. OpenAI was the lab that releases open models.
None of this is factually correct, that is it. I don't think this is debatable. I don't love OpenAI, but OpenAI made huge contributions to the field, and one should give credit where credit is due.
I have great trouble understanding why someone would waste time defending it.
I hope / think they are going to release more, just going for one big release a year like Gemma (if we talk strictly about general chat model -- Gemma 3 was March 2025)
Spreading propaganda through aligned model censored to eschew wrongthink? I mean, I truly believe there's some of that in the LLM world, but probably not the real reason you're searching for. Might be trying to (re)gain mindshare/cred amongst the hackers.
Is it though? Do we still have the expectation that LLMs will eventually be able to solve problems they haven't seen before? Or do we just want the most accurate auto complete at the cheapest price at this point?
It indicates that there's a good chance that they have trained on the test set, making the eval scores useless. Even if you have given up on the dream of generalization entirely, you can't meaningfully compare models which have trained on test to those which have not.
And the only people who could afford to tractor at scale are Cargill/Monsanto who bought out most of the small/medium-sized farms while leaving farms that didn't take the offer to slowly die...
And yet there isn't widespread unemployment. Fewer farmers were needed so fewer people became farmers. Food became cheap and plentiful. Everyone else went on to do other things that they couldn't afford to do before. Software will do the same; we will make more software with fewer people and it will become ubiquitous to the point that people will just quickly generate whatever software they need rather than do many monotonous tasks manually.
They're increasing reps and therefore total load. That's still a form of progression ('pushing yourself'). This style will slightly favor hypertrophy gains over strength gains.
At 40 I recently made this switch in style as well. The weight was getting so high that my anxiety was causing a mental aversion to working out altogether. Consistency is really 95% of exercise so I think this is a reasonable trade-off.
That said, I understand where you are coming from. There's something to be said about facing the fear of the weight head on. I've already done that in my younger years though. I'd much rather avoid injury and get 80% of the benefits.
You shouldn't be stressed of what's in front of you. Training also trains you for that other than muscle/power building. If you don't compete, you have no reason to be anxious. You should maybe dig into what's causing you that anxiety, if it's "I worry I won't make this weight", remind yourself that nothing will happen if you do, and if you do, it's part of the progression. I get this anxiousness also, but I always remind myself that.
I think that what you do in the gym will reflect on yourself.
Extensive tailwind training data in the models. Sure there's something more efficient but it's just safer to let the model leverage what it was trained on.
In my experience the LLMs work better with frameworks that have more rigid guidance. Something like Tailwind has a body of examples that work together, language to reason about the behavior needed, higher levels of abstraction (potentially), etc. This seems to be helpful.
The LLMs can certainly use raw CSS and it works well, the challenge is when you need consistent framing across many pages with mounting special cases, and the LLMs may make extrapolate small inconsistencies further. If you stick within a rigid framework, the inconsistencies should be less across a larger project (in theory, at least).
Professional legal services seem to be picking up steam. Which sort of makes sense as a natural follow on to programming, given that 'the law' is basically codified natural language.
I don't know how it is in other countries, but in the UK using LLMs for any form of paid legal services is hugely forbidden, and would also be insanely embarrassing. Like, 'turns out nobody had any qualifications and they were sending all the work to mechanical Turks in third world countries, who they refused to pay' levels of embarrassing.
I say this as someone who once had the bright idea of sending deadline reminders, complete with full names of cases, to my smart watch. It worked great and made me much more organised until my managers had to have a little chat about data protection and confidentiality and 'sorry, what the hell were you thinking?'.
I am no stranger to embarrassing attempts to jump the technological gun, or the wonders of automation in time saving.
But absolutely nobody in any professional legal context in the UK, that I can imagine, would use LLMs with any more gusto and pride than an industrial pack of diarrhoea relief pills or something - if you ever saw it in an office, you'd just hope it was for personal use and still feel a bit funny about shaking their hands.
While it doesn't seem we can agree on a meaning for AGI, I think a lot of people think of it as an intelligent entity that has 100% agency.
Currently we need to direct LLM's from task to task. They don't yet posses the capability of full real world context.
This is why I get confused when people talk about AI replacing jobs. It can replace work, but you still need skilled workers to guide them. To me, this could result in humans being even more valuable to businesses, and result in an even greater demand for labor.
If this is true, individuals need to race to learn how to use AI and use it well.
> Currently we need to direct LLM's from task to task.
Agent-loops that can work from larger scale goals work just fine. We can't letting them run with no oversight, but we certainly also don't need to micro-manage every task. Most days I'll have 3-4 agent-loops running in parallel, executing whole plans, that I only check in on occasionally.
I still need to review their output occasionally, but I certianly don't direct them task to task.
I do agree with you we still need skilled workers to guide them, so I don't think we necessarily disagree all that much, but we're past the point where they need to be micromanaged.
If we can't agree on a definition of AGI, then what good is it to say we have "human-in-the-loop AGI"? The only folks that will agree with you will be using your definition of AGI, which you haven't shared (at least in this posting). So, what is your definition of AGI?
They know that LLMs as a product are racing towards commoditization. Bye bye profit margins. The only way to win is regulation allowing a few approved providers.
They are more likely trying to race towards wildly overinflated government contracts because they aren't going to profit how they're currently operating without some of that funny money.
Yes, which is why the companies that develop the models aren't cost viable. (Google and others who can subsidize it at a loss obviously are excepted)
Where is the return on the model development costs if anybody can host a roughly equivalent model for the same price and completely bypass the model development cost?
Your point is inline with the entire bear thesis on these companies.
For any use cases which are analytical/backend oriented, and don't scale 1:1 with number of users (of which there are a lot), you can already run a close to cutting edge model on a few thousand dollars of hardware. I do this at home already
Open source models are still a year or so behind the SotA models released the last few months. The price to performance is definitely in favor of Open Source models however.
DeepMind is actively using Google’s LLMs on groundbreaking research. Anthropic is focused on security for businesses.
For consumers it’s still a better deal for a subscription than to invest a few grand in a personal LLM machine. There will be a time in the future where diminishing returns shortens this gap significantly, but I’m sure top LLM researchers are planning for this and will do whatever they can to keep their firm alive beyond the cost of scaling.
I am not suggesting these companies can't pivot or monetize elsewhere, but the return on developing a marginally better model in-house does not really justify the cost at this stage.
But to your point, developing research, drugs, security audits or any kind of services are all monetization of the application of the model, not the monetization of the development of new models.
Put more simply, say you develop the best LLM in the world, that's 15% better than peers on release at the cost of $5B. What is that same model/asset worth 1 year later when it performs at 85% of the latest LLM?
Already any 2023 and perhaps even 2024 vintage model is dead in the water and close to 0 value.
What is a best in class model built in 2025 going to be worth in 2026?
The asset is effectively 100% depreciated within a single year.
(Though I'm open to the idea that the results from past training runs can be reused for future models. This would certainly change the math)
For sure, all these companies are racing to have the strongest model, and as time goes on we quickly start reaching diminishing returns. DeepSeek came out at the beginning of this year, blew everyone's minds, and now look at how far the industry has progressed beyond it.
It doesn't even seem like these companies are in a battle of attrition to not be the first to go bankrupt. Watching this would be a lot more exciting if that was the case! I think if there was less competition between LLMs developers could slow down, maybe.
Looking at the prices of inference of open-source models, I would bet proprietary models are making a nice margin on API fees, but there is no way OpenAI will make their investors whole because they make a few dollars of revenue for a million tokens. I am terrified of the world we will live in if OpenAI will be able to reverse their balance sheet. I think there's no where else that investors want to put their money.
The other nightmare for these companies, is that any competitor can use their state of the art model for training another model. As some Chinese models are suspected to do. I personally think it's only fair, since those companies in the first place trained on a ton of data and nobody agreed to it. But it shows that training the frontier models have really low returns on investment
It is unclear. Everyday I seem to read contradictory headlines about whether or not inference is profitable.
If inference has significant profitability and you're the only game in town, you could do really well.
But without regulation, as a commodity, the margin on inference approaches zero.
None of this even speaks to recouping the R&D costs it takes to stay competitive. If they're not able to pull up the ladder, these frontier model companies could have a really bad time.
Yeah, but we can self-host them. At this point in the span of it, it's more about infrastructure and compute power to meet demand and Google won because it has many business models, massive cashflow, TPUs, and the infrastructure to build expanding on their current, which would take new companies ~25 years to map out compute, data centers and have a viable, tangible infrastructure all while trying to figure out profits.
I'm not sure about how the regulation of things would work, but prompt injections and whatever other attacks we haven't seen yet where agents can be hijacked and made to do things sounds pretty scary.
It's a race towards AGI at this point. Not sure if that can be achieved as language != consciousness IMO
Who is "we", and what are the actual capabilities of the self-hosted models? Do they do the things that people want/are willing to pay money for? Can they integrate with my documents in O365/Google Drive or my calendar/email in hosted platforms? Can most users without a CS degree and a decade of Linux experience actually get them installed or interact with them? Are they integratable with the tools they use?
Statistically close to "everyone" cannot run great models locally. GPUs are expensive and niche, especially with large amounts of VRAM.
>It's a race towards AGI at this point. Not sure if that can be achieved as language != consciousness IMO
However it is arguable that thought is relatable with conscienceness. I’m aware non-linguistic thought exists and is vital to any definition of conscienceness, but LLMs technically dont think in words, they think in tokens, so I could imagine this getting closer.
'think' is one of those words that used to mean something but is now hopelessly vague- in discussions like these it becomes a blunt instrument. IMO LLMs don't 'think' at all - they predict what their model is most likely to say based on previously observed patterns. There is no world model or novelty. They are exceptionally useful idea adjacency lookup tools. They compress and organize data in a way that makes it shockingly easy to access, but they only 'think' in the way the Dewey decimal system thinks.
if we were having this conversation in 2023 I would agree with you, but LLM's have advanced so much that they are essentially efficient lookup tables is an oversimplification so dramatic I know you don't understand what you're talking about.
No one accuses the Dewey decimal system of thinking.
If I am so ignorant maybe you'd like to expand on exactly why I'm wrong. It should be easy since the oversimplification is dramatic enough that it made you this aggressive.
I'm not the other poster but he's probably referring to how your comment seems to only be talking about "pure" LLMs and seems pretty out of date, whereas most tools people are using in 2025 use LLMs as glue to stitch together other powerful systems.
The bottleneck for commoditization is hardware. The manufacture of the hardware required is led by tmsc and samsung being a close second. The tooling required for manufacture is centralized with ASML and several other smaller players like Zeiss and the design of the product centers around nvidia though there are players like AMD who are attempting to catch up.
It is a complex supply chain but each section of the chain is held by only a few companies. Hopefully this is enough competition to accelerate the development of computational technologies that can run and train these LLMs at home. I give it a decade or more.
Another way to win is through exclusive access to high quality training data. Training data quality and quantity represent an upper bound on LLM performance. That's why the frontier model developers are investing some of their "war chests" in purchasing exclusive rights to data locked up behind corporate firewalls, and even hiring human subject matter experts in order to create custom proprietary training data in certain strategic domains.
That's a good line but it only works if market forces don't commoditize you first. Blithely saying "commoditize your complement" is a bit like saying "draw the rest of the owl."
Free models given away by social media companies (because they want people to generate content) and hardware companies (because they want people to buy GPUs, or whatever replaces them). Can the current subscription models compete with free? It's just a prediction - it could well be wrong.
That would be true in a monopolistic market. But these frontier models are all competing against each other. The incentive to 'just work and get shit done fast' is there as they each try to gain market share.
Google is the only USA based frontier lab releasing open models. I know they aren't doing it out of the goodness of their hearts.
reply