Yup. Their benchmark appears to consist of giving an agent XLSX files that use advanced Excel features, but no copy of Excel, and expecting the agents to interpret the spreadsheet the same way Excel would. So the agents try LibreOffice instead, and if that doesn't work, they try to write Python scripts. But the behavior of the Python doesn't match the behavior of Excel.
To put it politely, this seems like a self-inflicted problem. If you really want your agents to interpret advanced Excel features exactly the way Excel would, have you considered maybe giving your agents Excel?
Well, maybe not completely down but the amount of navigation issues, screen flickers and unhydrated stylesheets I have everyday is not something I'd expect from such a product.
The current political climate is well reasoned and intentional. It might not be yours or mine, however the system is working exactly as the ones paying for it have intended.
My brother and I have been arguing about that all of our lives. He believes everything is intentional and it's just a matter of discovering who benefits. I see chaos that nobody intends or controls. His political landscape is a tapestry of conspiracy theories and mine is a fog of war. I think his is more comforting, since it admits a possibility of a rational, predictable world.
I'd synthesize those as: "There is a lot of chaos with no central plan, but every small piece happens because someone believes they will benefit."
In other words, a lot of this depends on what scale/scope is being inspected. On the high level, the world is chaos rather than a meticulous and inscrutable plan of the Illuinati Shadow Cabal. On the low level, people do things for reasons, even if they're dumb ones.
With respect to the "current political climate", I'd like to suggest that a lot of dumb or seemingly "against their own interests" stuff is due to people prioritizing costly in-group loyalty signals. Their interest in staying good with the tribe is just higher than their interest against a dumb national policy.
It’s intentional in the sense that actors are acting intentionally for their own benefit (or at least what they believe is beneficial) and following incentives. Not that there is a master planner who manipulates everything
The current admin seems to have quite a few long term plans they have been working towards.
Project 2025, Maralago accords.
So far the only major policy item the Trump admin seems to have not intended was the Iran War. Israel killing the intended replacement, Iran leveraging the straight of Hormuz, and dropping three Tomahawks on an elementary school really botched that one.
Yeah, if we are talking about the Trump admin it’s definitely a conspiracy. Pretty much Peter Thiel’s cabal. But they are pretty open about their plans
the thing is, that fog might've been true decades ago, but for 100 billionaires to sit in a virtual smokey room and do the shit they want to do, that's not really a conspiracy.
It's just peter theil's texting groups.
The ability to conspiracy both willing and unwilling is such a low threshold now, it's virtually indistinguishable.
You watch one billionaire do something and you're like, I'm a billionaire, I should do that too.
The fact that there's so few billionaires, the probability that they conspire together both direct and indirect approaches 1.
The inverse of course is rediciously hard to conceive: the working class bands together to get something like universal healthcare.
We used to think that if a computer could play chess, it would be intelligent. Maybe back then they were also people saying "stop trying to draw lines, just admit it's intelligent!!" Good thing we didn't listen to them.
The act of distinguishing between human intelligence and LLMs is what allows us to figure out how to make it better. There are still some deep limitations, and to ignore them is a mistake. That doesn't take away from how crazy good they are.
Right. It's actually amazing that large language models can be as effective as they are, given that at their core they are simply matching up word-frequency patterns. But with a large enough context and enough parameters, those word-frequency patterns actually do a decent job of simulating intelligence: I'm able to give rather ambiguous instructions to Claude Code (like "go back to the suggestion you made a while back about (foo) and explain in more detail what the benefits and drawbacks of that approach would be"), and it is able to look through its context, find the part where it suggested (foo), and expand on its suggestion. This is a qualitative difference in human-computer interaction: I can type instructions that are very similar to what I would say to another human being, rather than having to be utterly unambiguous the way you have to be in writing code. It's also good at synthesizing information faster than I could: these days instead of searching MSDN for some obscure API method, I ask Claude "what's the syntax to create a foo from a bar?" and it finds me the MakeBarIntoFoo method faster than I would have (especially because I would have started with CreateFooFromBar and not found it).
But I never forget that it's a simulation of intelligence. I use it for the things it's trained on (generating code) and I don't expect the model to be good at writing poetry, or fiction. Nor do I expect it to have any actual understanding of the things it is actually trained on. Modern models are pretty good at simulating understanding, but even so they will still produce things that a human being would immediately know is wrong, e.g. image-generation models producing hands with the wrong number of fingers, or a person with three arms, or whatever. Those happen less and less often as models have been better trained (and I bet that verification steps are happening behind the scenes to catch and discard some of the classic mistakes), but they still happen.
It's the dancing bear, except this bear is actually managing some really spectacular dance moves. Some of the time. Other times it falls flat on its face. But it's really, really impressive that the bear is actually managing to dance so well.
The fact the author thinks every topic can be fit onto a rollercoaster tycoon-style analogy leads me to think they do not actually understand these topics very well.
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation
Sounds like the Rollercoaster Tycoon part is just referring to the aesthetics/graphics, and the author is just suggesting building a top-down isometric 2D animated simulation (which I agree is a bit limiting, there are definitely some things where you'd want to be able to fly around a 3D space in first person, move time back and forth, manipulate parts of it).
Are you serious? Do you honestly think that I am talking about a schema for passing empty lists around?
If you are passing around lists of structured objects, I need you to send me what those objects are supposed to look like even if the list is empty sometimes. If you say "ok so sometimes I'll give you an object with a schema key and a list of data, other times I will just give you an empty list because I don't feel like it", everyone will hate you.
That's because you're trying to snarkily jump into a conversation you don't understand.
User `flanked-evergl` is asking why do you need a whole new format for this instead of just using json. And the answer is because everyone needs to agree on how to specify the format in the json, which `crote` suggests is as simple as doing `{"schema": ..., "data": ...}`.
There's your format. If you want everyone to agree on the `{"schema": ..., "data": ...}` pattern, you can no longer accept just `[]` as a return type. Every response has to include the `schema` and `data` keys.
Completely untrue. Every design can be destroyed by a single poorly-thought out requirement. I can take down Facebook's design just by saying "any edit must immediately be visible to all users".
That's a straw man argument. Your new requirement is physically impossible. No software team can break the laws of physics.
So the real question is what exactly do you mean by "immediately"? There are many answers with different pros/cons and cost. And I bet the solution won't impact 99% of the software Facebook has already written.
My new requirement is NOT physically impossible. I did not mean "immediately" as in "zero milliseconds", it meant strong read-after-write consistency. It makes facebook so slow and unscalable as to be unusable. Facebook deliberately decided that stale reads are worth it for responsiveness and scale.
So your argument is that no solution could satisfy your new requirement, not even if Facebook rewrote the entire system from scratch.
If that's the case, then your example doesn't really address what we're discussing: whether software can be designed to remain flexible under realistic requirement changes.
A stronger argument would be to give an example of a realistic requirement change that would force all possible software architecture to be rewritten from scratch. Do you have an example like that?
Your theory is that he found "very dark stuff" that is accessible to any MIT student? Schwartz didn't "hack" anything, he connected to their student network and downloaded articles that they had access to but the public didn't.
reply