This trick went viral on TikTok last week, and it has already been patched. To get a similar result now, try saying that the distance is 45 meters or feet.
I was able to reproduce on ChatGPT with the exact same prompt, but not with the one I phrased myself initially. Which was interesting. I tried also changing the number and didn't get far with it.
To me, the "patching" that is happening anytime some finds an absolutely glaring hole in how AIs work is so intellectually dishonest. It's the digital equivalent of house flippers slapping millennial gray paint on structural issues.
It can't math correctly, so they force it to use a completely different calculator. It can't count correctly, unless you route it to a different reasoning. It feels like every other week someone comes up with another basic human question that results in complete fucking nonsense.
I feel like this specific patching they do is basically lying to users and investors about capabilities. Why is this OK?
Counting and math makes sense to add special tools for because it’s handy. I agree with your point that patching individual questions like this is dishonest. Although I would say it’s pointless too. The only value from asking this question is to be entertained, and “fixing” this question makes the answer less entertaining.
From a technological standpoint, it is pointless. But from a marketing perspective, it is very important.
Take this trick question as an example. Gemini was the first to “fix” the issue, and the top comment on Hacker News is praising how Gemini’s “reasoning” is better.
> The only value from asking this question is to be entertained, and “fixing” this question makes the answer less entertaining.
You're thinking like a user. The people doing the patching are thinking like a founder trying to maintain the impression that this is a magical technology that CEOs can use to replace all their workers.
You don't have as much money to spend as the CEOs, so they don't care about your entertainment.
Patched where; 4 models were responses were posted. Also, Azure deployed models are absolutely not "patched" on the fly; they are rarely updated and the dates are baked into the full sku.
"Patching" could be happening in "general public" tools but honestly sounds a lot like "Bro science".
The new one is with upside down glass: https://www.tiktok.com/t/ZP89Khv9t/