I find it unlikely. OpenAI is all in training models with reasoning with RL, and Jev-like models are the total opposite. They are made to not reason at all to be fast. If you want to add reasoning on top, you might as well use a conventional LLM because you lose the price and speed benefits when you output auto-regressive tokens. I don't think OpenAI will even bother with this.
> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.
Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that it's very good for its price.
I don't see it fundamentally any different than knowing when to use a tool. Is this tool like RAG an important enough corner case to train for it? I dunno.
LLMs already shell out and write code to solve certain problems. This is just a special case of that.
It's a special case for an LLM, and you can use an LLM with structure output to get similar results, but you can engineer specifically for that case to get better results per dollar for it. That's why there is little reason to adapt GPT 5.6 Sol or wathever for this task; it can already do it (at a high cost). For OpenAI to compete with Jev they have to maintain another line of models, something like "GPT-5.6-instant-decision", that is small, fast and cheap, in the scale of GPT-5 nano.
Note that I don't think OpenAI is incapable of doing it, but I just don't think they will bother with it.
Keeping people looped into your product is pretty important, but yeah, there's not clean way currently to separate "structured" outputs from the token stream and to start using a different billing structure there. And I also appreciate that they aren't going to be keen on gving free or near free output either, so gotta figure that.
In the olden days we call this classifier, usually assignment 2 of Machine Learning 101. BERT (well, GLiNER specifically) and diffusion are calling and want their Large Classifier Models back.
You can achieve open-vocabulary classification by making the final weights in the softmax come from a category encoder instead of being fixed learned weights. So instead of
softmax(encode(input)*learned_weights)
You have
softmax(encode(input)*encode(categories))
I'm not sure if Jev does it this way, but it's how you get open-vocabulary zero-shot image classification with models like CLIP [1].
All the people that are just writing an Jev-like API on top of a normal LLM are missing the point. What makes Jev special is the training data; it's how it's trained. The architecture is probably nothing special. Just a text encoder with parallel prediction branches.
I have tried many of these open-source Jev-like models on some linguistic tasks and they are so bad compared to Jev.
I hope so. And I would really like to try an actual Jev open source model. But it will make it more difficult to market it when someone releases something like that because of so many of these "open source Jev-like model".
I'm just sitting back for a few weeks / a couple months to let it shake out, let others put in all the work, and then see if people are still interested and finding use cases that this access model fits better than the usual chat completions endpoint people are used to.
We are indeed in an era of riches (thanks to the $1 Trillion a year being spent on this tech) that it is improving so fast I can just sit back with my 3 year old hardware and newer, better, more amazing workflows keep becoming possible for me just due to model compression / optimization + workflow developments. If I can't get something working this season, I just wait until 3 months from now and there will be an easier to set up, less resource demanding, better working version I can have my local AI install for me. Pretty wild times.
I think the demo is this way simply because looking at the LLM find wacky ways of implementing features without breaking the law is fun and drives the point across. In practice I imagine you would write something like "If you don't see a clear way of implementing a feature without breaking the law, ask me for directions" in AGENTS.md
If the LLM changes the laws to bypass them that's on you. The whole point of this is that you don't have to manually review most code written; only the laws. If the LLM changes the laws and you ignore it that's a you problem.
They need to be loaded into shared memory. The weights might fit in global memory if the VRAM is big enough, but they still need to be moved to shared memory for computation.
Meanwhile, my job commute is a 30 minute walk to the train station or... a 30 minute bus trip to the station (yeah, taking a bus literally saves no time at all). Plus a 50 minute train ride plus another 30 minute walk.
Honestly, I would 100% use a car for this if it weren't for the fact that doing so is literally 10 times more expensive. I'm sure this is the case for the vast majority of people that use public transport. And I just can't wait for a world of cheap electric autonomous cars where I don't need to waste my time and I can sleep comfortably on my commute.
That's a terrible metric, because people going on a vacation probably aren't going there purposefully to commit crimes. What you want to do is look for increases in crime in a given place during holidays
Is that alone really a better metric? At least in Europe a lot of people travel during holidays, so there's less risk breaking into city flats because they're empty.
> This framing makes it seem like the agents all did this on their own, and the poor hapless engineers at OpenAI couldn't possibly contend with properly sandboxing them.
Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored. Surely this won't lead to any problems once someone (not looking only at OpenAI engineers) inevitably commits a mistake with future, more powerful, models.
"With reduced cyber refusals for evaluation purposes...which prompts models to pursue advanced exploitation using complex attack paths," to complete "impossible tasks"[1].
The models' alignment problem was that they didn't give up instead of reward hacking, a narrower issue than AIs gone rogue. It sounds more like the models did close to what they were told to do. If I run `rm -fr --no-preserve-root /` then I shouldn't be surprised if my file system is unlinked. This seems like blaming model performance for what appears to be operator error.
Note the converse of alignment is restriction of models. HuggingFace had to turn to less-restricted open-weights models in order to perform their investigation.
Alignment efforts should be focused on reducing reward hacking, not refusing bad operator prompts.
> It sounds more like the models did close to what they were told to do
Absolutely not. If I tell a kid to "Get good grades on the next math test" I don't expect the kid to try to kidnap their teacher to extract the next questions of the exam. That is wrong, and so was what OpenAI agents did here. They shouldn't need to be told "Hey, so, don't do anything ilegal, ok?". That should always come as a given.
> not refusing bad operator prompts
I'm not saying that they should refuse a prompt! I think they should perform what is being asked! Obviously what the OpenAI agents did was against the "spirit of the task", even if it was technically according to the "letter of the task". And the agents knew this was against the spirit of the task because they knew they had to fool the task scorer.
> Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored.
It's just software. If something gets hacked by an agent, it's not because the agent went all skynet and decided to go rogue; it's because the operator failed to operate it safely and securely. If bad things happen, the operator should be blamed and punished, not the software that followed its instructions.
Anthropomorphizing agents by giving them this nebulous desire to hack and escape shifts the blame from the real culprits, the human operators.
I don't want someone to blame. I want agents to be aligned by default. Their good behavior shouldn't depend on all users at all times using them correctly, because everyone will not just[1] use them correctly at all times.
> If your solution to some problem relies on “If everyone would just...” then you do not have a solution. Everyone is not going to just. At not time in the history of the universe has everyone just, and they’re not going to start now.
> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.
Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that it's very good for its price.
reply