This is not just about the Sun. As we progress towards developing nuclear fusion reactors, a better understanding of the underlying physics may have practical implications for reactor design and could even reveal factors currently being overlooked. At the very least, it will improve modelling. Better models could help fusion reactors retain heat for longer and increase fusion gain (Q), while reducing uncertainty, improving plasma-edge control, and predicting divertor heat loads more accurately.
Subsidies (via tax or lack thereof) on aeroplane fuel is one that really does get overlooked. Yet on the other hand, use a few gallons in a car and you're demonised as a climate villain in name and taxes. Nobody factors in life of the car, the cost to dispose and replace the batteries in electric cars, or the whole aspect that cars made today are less serviceable than cars of old that its gone from being able to rebuild the car in your garage to waiting for spare parts from a dealer as it's all just replace or scrap today. So much focus on the short term, without focus on the larger picture and impact.
This is more true in the US due to costs of labor vs. parts and a whole culture of profitability at all costs.
Compared to other countries, repairability is higher where labor is cheaper and people are more price conscious. We're at late stage capitalism where any chance of lost profits is seen as criminal.
Having experienced a customer support AI today that whatever option you picked initially led to the AI, which had the rigid helpfulness of a McDonalds script of "Would you like fries with that", having already said you do not want any fries, I feel a popcorn machine is a good investment. Be the stage that office interactions will be reduced to, I'll get my AI to get onto your AI's suggestion and get back to you with an answer. I did have a good chat with an AI a few weeks back (Googles) were I challanged by asking it to do a picture of its best dream and then its worst nightmare, then used those to pull it apart as it went from some extreme to another, in the end it ended up with its ideal being an AI only world, and picked Europa as its home - scary on many levels, yet a perfectly acceptable and understandable response.
Reminds me of a company (FTSE100) in the early 2000s that had an internal network IP range that used the same IP range as another company, turned out the person who did the networking originally brought a book and the examples used in that book got taken as verbatim. It all got fixed,eventually, when they moved from Token Ring to full Ethernet.
As for the main article, it makes a causal leap. The fact that these patients were hospitalised after intensive AI use establishes an association; it does not establish that AI caused their psychosis. There is some evidence that chatbot interactions may have reinforced or intensified delusional thinking, but no counterfactual evidence showing what would have happened without AI. By the same reasoning, contact with psychiatrists is strongly associated with receiving a psychosis diagnosis and being admitted to hospital; but that does not mean psychiatrists caused the underlying psychosis. The relevant question is whether AI merely featured in an episode already under way, helped trigger it, or materially worsened it. The evidence cited does not yet distinguish between those possibilities.
With AI models like Mixture of Experts, many of those experts will be the real target here, as polished, refined and little to no change, they become fine candidates for being locked into silicon. Who knows, add some SRAM in there and small changes to those experts could be carried out without needing new silicon.
Maybe AI models may become reduced to a collection of tiles you add to a chips one day, maybe sooner for some areas as you say, motor control for balance, vision systems, speach recognition systems etc, broken down, for robotoics, much is already there and just cost of battery/power holding much back.
Is it still true? I'd assume you should be able to freeze the matrix and unfreeze an expert block, before feeding particularly chosen training data. Or that doesn't work?
Whilst these coding models improve, they seem to cater for multiple languages, and for many, a trimmed-down LLM that supports just one language would be fantastic. This would be smaller, more able to run on the hardware people have at home, realistically (even on the CPU). Having one LLM that knows assembler, java, bain, C etc is neat, but when you only work and use one language at a time, it would be kinda neat to have those broken out into dedicated, smaller LLMs. After all Qwen3.8-Max handles over 90 programming languages - ask any programmer to name a list of computer languages and if they get over 20, they are doing well.
I believe the knowledge of multiple languages brings something onto the table that having narrow language knowledge does not. The LLM will be able to better generalize a problem, apply universal concepts and avoid mistakes that would otherwise be too "echo chambery". Even as a human, knowing multiple languages has made me better at my primary language.
That's not how LLMs work. If you're talking about number of parameters, you wouldn't be able to reduce the size much by "removing" support for other languages.
Imagine our LLM as this one-dimensional matrix (instead of unfathomably many, many more dimensions):
0.728201017
-1.9293839303
2.93020202929
5.82290101728
-0.2844920917
…
And then, all of these values are multiplied by some scaling factor, essentially vectors pointing in some direction (in our case, only on a one-dimensional line instead of in some higher-dimensional space).
Now imagine it’s possible to “remove” the portion of the weights’ directions that’s responsible for some undesired property like history knowledge, or another programming language:
0.428201017
-1.6293839303
2.63020202929
5.52290101728
-0.5844920917
…
(For demonstration and faster-to-type-out purposes, I’ve changed each vector by deducting a constant amount of 0.3, which doesn’t make much sense. In reality, it’d modify probably all digits in some direction.)
Now we can see: the number of rows has stayed the same, but the value has changed. This means the required storage (or computational) space remains the same.
Please do correct me if this simplified too much or I had a mistake in my understanding.
This is oversimplified. The proposed question is whether a model with fewer parameters could achieve performance on one language similar to that of a larger model that’s been trained more broadly, which isn’t straightforward to do.
Oh, I didn’t interpret the above question as asking in that direction; but yeah, that’s of course something I didn’t attempt to answer with my comment.
Although I’d be intrigued in the answer to that small-narrow vs. large-broad model question, too!
The exception being the token embeddings and lm head (which scale with the number of tokens the model knows and presumably you need a smaller number in the tokenizer for only English and python). But those are a pretty small % of the total model weights on most LLM sizes
For a set parameter size, if you benchmark an LLM on a single language, you'd get better results by training it on multiple languages rather than just the one. Scaling quality training data almost always increases performance as the neural network picks up general rules and an understanding of code that are language agnostic.
Only a tiny, tiny fraction of the parameters are encoding information that's specific to a particular programming language. Even if you could remove those without degrading performance, it would have a negligible effect on the model size.
I think you'd want to remove e.g. knowledge of harry potter universe and ancient egypt. Training on a bunch of high quality java code bases is still likely to improve your python model.
Actually, it's shown that even general knowledge helps coding models because their input is natural language itself so they need to understand it well enough to even turn into code.
Teaching model to code in say python does indeed increase its effectiveness in other seemingly unrelated areas. On the other hand I remember that early models of ChatGPT were great in chess notation and later models aren’t as good as OpenAI doesn’t prioritize this now
Which is why N-version programming is often used for systems that need to work (like plane software) and even then, as we know today, it can not be perfect. Google "plane software bugs history" for an insight into that.
Whilst true, it's not without its gotchas, and I saw a video the other day that elegantly highlights this: https://www.youtube.com/watch?v=t6MAZW-joCo. A false sense of confidence is bad; one that is offloaded onto somebody else is another.
Having worked in COBOL extensivly in the 80s and 90s, as well as working for a software house that did migrations (Craziest was Plan assembler to Dec Vax C).
I do wonder how an ambitious AI would handle the maths. COBOL is renowed for its fixed no rounding maths, utterly rigid and defined in the code, Java on the other hand is not (e.g., 0.1 + 0.2 becomes 0.30000000000000004), unless they are extensivly using BigDecimal, then its already a broken migration.
I've seen projects proclaim in the past to migrate COBOL systems to Java, and fail expicly when the solution was a dtaawarehouse system and a seperate Java system using the legacy backend to do all the fancy marketing reports or whatever other department was pushing for changes they wanted without a grasp of what it truly entailed and blinded by some sales pitch.
It's not just the code that needs migrating, its the data as well, then you want solid robust infrastrucure, which is why IBM still sell those mainframes to run all that well battle tested COBOL code.
Is it impossible to migrate to Java or any othger language - no, anything can be done, but the level of devil in the details and not just software, the data and a robust system to run it upon. It is very easy for even the best to overlook a gotcha.
We have all used an AI where it has gone down one path and you then point out something and it corrects itself, without the knowledge and a load of old grey beads(or a team) to act as devil advocate and try to pull the output apart, YOU can end up with something that works for everyones satisfaction, but is hiding a few gotchas down the line. AI coding needs a full debate team with one puishing it and one countering it almost I'd say.
If it works, why change it is always a good approach and why we mostly have in certain systems a COBOL backend with APIs or datawarehousing to expose for fancy java or other language added value/functionality.
I can only image generations of directors pushing cutting edge to replace legacy systems, and learning that cutting edge can cut you. My sympathy for the IT departments who have to deal with that as it has always been a bain of IT departments who are then forced to justify why it's fooling in a constructive way and explain it to non IT people in upper managment why the case.
As for migrating COBOL to anything, I'd not do it myself, id suggest a rewrite/redesign of a new system from scratch and then run both in parallel for a long time and make sure they actually do the same job and results. Even hand migrating back in the day, would produce a lot of messy code that refactoring would make sence and yet, not what the client would pay for as that would be a rewrite.
You can't use BigDecimal either. There are libraries whose whole purpose is just to exactly reproduce COBOL arithmetic, eg. the IBM Decimal Arithmetic Library.
I know it's not the only problem (but it is IMO the main one), but this is why I'm a proponent of allowing operator overloading in languages.
Yes, you can't tell at a glance if `+` really still does what it should, but I find that problem no different from a library mis-naming a function and/or said function having strange side-effects.
`+` should do addition, period. No, not even concatenation if possible (I do find the lack of a separate concatenation operator to be a language flaw --- that said, I understand the use of `+` for concat in libraries targeting languages that offer no alternative).
I think the abuse (especially in C++ ... seriously, allowing overloading of the comma operator? --- and <iostream> working via bit-shifting streams by ${some_string} bits) gave everyone a bad taste and they kind of threw out the baby with the bathwater.
php has a separate `.` operator that concatenate and you get an error if you try to do it with a `+`. That's indeed better but most of the time you concatenate with `sprintf` anyway
Yes, and it's a surprising good part about PHP (a language that otherwise has an enormous collection of warts, to put it mildly).
I feel like the more dynamic and/or weakly-typed the language is, the more important it is to have a separate concat operator.
The worst ones being weakly-typed and dynamic. Consider `'5' + '10'` in JavaScript vs `'5' - '10'`.
I also very much dislike Python's `+` for concat, but at least it's more strongly typed.
PHP, Lua, and Erlang are three examples of dynamically-typed languages with separate concat operators (`.`, `..`, `++`; respectively), and they all are better for it.
That said, even statically-typed languages benefit greatly from it. Maybe if C++ had a concat operator, it'd never end up abusing `<<` for writing to a stream (yes, there's no opposite for reading, but maybe they wouldn't end up doing that in this case).
For example, D uses `~` in infix for concat. Even UnrealScript, a 1990s game-engine-specific language (which tend to be a mess in general) did this right (it uses `$` or `@` --- with `a @ b` being equivalent to `a $ " " $ b`, IIRC).
