Many comments seem to be misinterpreting this, but all this seems to be suggesting is that the AI companies should not be treating mathematical milestones as benchmark targets but need to release the cumulative findings in a human interpretable and digestible way that enriches and develops the field. See Thurston quote on Tao's banner:
> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand
and think more clearly and effectively about mathematics.
The comment under “Future Directions” at the end of that paper would seem to apply to the use of AI in software development, business management, and other areas as well:
“For AI[-assisted mathematics] research, improving research judgement and research-state representation may require training and evaluation on records of mathematics as a process, including failed approaches, strategic decisions, and evolving assessments of evidence. For mathematical practice, while these limitations persist, human expertise is likely to remain most valuable in these global functionalities: deciding when to persist or reframe a direction, and maintaining an accurate representation of accumulated progress.”
Yeah, the end goal is that you can run the economy autonomously, at higher levels of complexity than people could understand or reasonably make decisions about.
This makes no sense. It would be smart enough to understand but not enough to explain it, or are you assuming the human brain can’t comprehend the world by default?
The section where the Research-state representation is assessed to be fragile is the work I want to see more of.
I've already accepted that models know everything and will only get smarter, its cope to pretend hallucination is achilles heel. I want to understand how to build/use harnesses that converge on goal states and allow me to contribute human expertise like intuition and taste.
The components they call bulletin, session report, and especially the curated summary are the parts that make their work go forward.
Because those models already exist and can be run on consumer available hardware with no real issues. All this does is create barriers for Anthropic competitors.
Michael Edward Johnson has an interesting theory called vasocomputation with the core hypothesis:
> vasomuscular tension stabilizes local neural patterns. A sustained thought is a pattern of vascular clenching that reduces dynamic range in nearby neurons. The thought (congealed pattern) persists until the muscle relaxes.
This person is part of a group that seemingly posts to twitter and cites twitter as statement against academic institution and traditional scholarship. Some members of the group are academics themselves. Maybe some things can only be posted there, and so your intuition is correct. But I think you misunderstand why specifically you are seeing twitter links so prominently used. My sense is it's attacking the politics of citation.
If that's true, that might also be part of why dreams exist. If the glymphatic cleanout during sleep is a rhythmic constriction wave across the brain, then whatever the brain is doing at the time the wave crosses it becomes a sustained thought as a side effect.
Maybe that's why sleep, in the sense of becoming immobile, is a thing. The brain disconnects a lot of the motor control so that the hallucination caused by sustained randomness from cleanout waves don't make us flail all over the place and hurt ourselves.
You could, but in many cases you wouldn't want to. You will get superior results with a fixed compute budget by relying on external tool use (where "tool" is defined liberally, and can include smaller narrow neural nets like GraphCast & AlphaGo) rather that stuffing all tools into a monolithic model.
Isn't that what the original resnet project disproved? Rather than trying to hand-manicure what the NN should look for, just make it deep enough and give it enough training data, and it'll figure things out on its own, even better than if we told it what to look out for.
Of course, cost-wise and training time wise, we're probably a long way off from being able to replicate that in a general purpose NN. But in theory, given enough money and time, presumably it's possible, and conceivably would produce better results.
I'm not proposing hand-engineering anything, though. I'm proposing giving the AI tools, like a calculator API, a code interpreter, search, and perhaps a suite of narrow AIs that are superhuman in niche domains. The AI with tool use should outperform a competitor AI that doesn't have access to these tools, all else equal. The reason should be intuitive: the AI with tool use can dedicate more of its compute to the reasoning that is not addressed by the available tools. I don't think my views here are inconsistent with The Bitter Lesson.
yep, also i think while they could have issues with dataset sizes less than 2^k, it's interesting to note their use in accelerating clustering algos like dbscan. they do make neat visualizations though https://marimo.app/?slug=x5fa0x
> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.
reply