I agree with your points. My problem is that I don't see any way to "stop" AI development at a level just below human capability where you get most of the benefits but still have humans firmly in control.
Regarding entertainment jobs like sports star, musician, artist, actor, etc., the problem is those are professions where, for the most part, only the very best can make a living. Nobody is that interested in seeing the matches of the 324th best tennis player in the world, despite that person being in the top .01% of tennis players globally. Similarly, only a tiny percentage of actors "make it", everyone else does it for a few years and then leaves because they enjoy eating, or they accept poverty wages for a long time.
I get pretty scared when I hear the AI utopianists rattle off all the benefits that are coming (and when they play down the risks) because it feels clear to me that they really haven't wargamed out what society would look like with very powerful AI.
Regarding entertainment jobs like sports star, musician, artist, actor, etc., the problem is those are professions where, for the most part, only the very best can make a living. Nobody is that interested in seeing the matches of the 324th best tennis player in the world, despite that person being in the top .01% of tennis players globally. Similarly, only a tiny percentage of actors "make it", everyone else does it for a few years and then leaves because they enjoy eating, or they accept poverty wages for a long time.
I get pretty scared when I hear the AI utopianists rattle off all the benefits that are coming (and when they play down the risks) because it feels clear to me that they really haven't wargamed out what society would look like with very powerful AI.