I love the culture books too but c’mon man.
First, how do you imagine transferring power to an AI overlord would sit with the people who currently have all the proverbial guns in our current society. That’s the problem we need to solve first.
Second, who in our society would you trust to build an AI up to the task of single handedly managing world affairs? How would you even test something so superhuman? Not to mention that we’re not close to superhuman, we’re already pushing the limits of our tech building a chatbot that can’t stay coherent for more than a couple paragraphs. The tech just isn’t there yet and not within orders of magnitude.
It’s like that for many vices such as alcohol- it’s harmful but banning it is much worse than regulating it (see the history of 1920s America). I’m not convinced porn consumption is harmful but even if it was, it shouldn’t be banned.