Op was clearly not talking about Alpha Zero, a different technology made by different people for a different purpose. Instead, they were noting that despite displaying some truly excellent world modeling, GPT-3 is trained on data that encourages it to vomit up rehashes. It's very possible that the next generation will overcome this and wind up completely holding together long-run concepts and recursion, at least if scaling parameters keeps working, but for now it is a real limitation.
GPT-3 writes like a sleepy college student with 30 minutes before the due date; with shockingly complete grasp of language, but perhaps not complete understanding of content. That's not just an analogy, I am a sleepy college student. When I write an essay without thinking too hard it displays exactly the errors that GPT-3 makes.
GPT-3 writes like a sleepy college student with 30 minutes before the due date; with shockingly complete grasp of language, but perhaps not complete understanding of content. That's not just an analogy, I am a sleepy college student. When I write an essay without thinking too hard it displays exactly the errors that GPT-3 makes.