Logical conclusions are only as valid as your model. Whenever I see people praising their own logic, I hear "My axioms are so perfect that I'm not capable of questioning them, or even knowing what they are."
Absolutely. These folks always remind me of Robert Brandom's* idea of "formal logical inference" vs. "material logical inference." The former focuses only on the formal structure of an argument where the latter takes into account context and other variables.*
* I don't know if the idea originated with Brandom.
* Please, forgive the crude (partially wrong?) explanation.
It’s reasoning without reality, the same mocked with sayings like “arguing about how many angels can fit on the head of a pin.”
Reasoning must be checked against observation and experiment. It must be checked against reality.
It’s not just that humans are fallible and full of biases and failure modes. It’s that reason itself is only capable of crystalline perfection in domains where all priors can be enumerated and the system is closed, like pure math. Even there we know of mathematical constructs that exhibit phenomena like emergence and computational irreducibility where the future state can’t be guessed without fully evaluating the function; where “leaps” are provably impossible.
The craziest stuff is some of the longtermism stuff. They are literally writing sci fi (and cliche sci fi at that) and then reasoning from it as if it were real and using those conclusions to guide present day moral and political thinking. Absolute lunacy. We can’t predict the next 50 years let alone the next 5000.
Sometimes I think one of the evolutionary counter pressures that has likely prevented human IQ from being driven entirely up and to the right is that high IQ increases propensity for delusional thinking. I’ve met quite a few incredibly brilliant idiots. The smarter you are the more elaborate a prison you can construct for your own brain. The major innovation of the discipline of science (and it is a discipline) was to put forward a method to avoid this by taking a step, confirming, and only then taking the next step. It’s pretty simple but it requires restraint even when there’s a shiny thing that looks so “truthy” and cool.
Agreed, but a question occurred to me: what if high IQ people aren't more delusional but more capable of articulating or acting upon their delusions, thus making it more apparent?
The delusions of a normal intellect are probably less interesting, and they're less likely to be a person who garners attention generally. But they may be equally detached from reality.
High IQ in a modern test is about pattern recognition.
I haven't seen it pointed out yet, but oversensitive pattern recognition can lead to illusions.
I.e. if you didn't sleep for two days, you can start "seeing" weird stuff sometimes. It's cause you brain is hallucinating something that isn't there, but because you see something, brain recognizes pattern and alerts you before checking against other patterns and common sense.
So I'd imagine that if someone is very good at pattern recognition they could have very high IQ but also not have enough erudition to check if the patterns they see is actually there.
Which arguably what could have happened in Yudkowskis case, since he scored very high on IQ, but dropped out of middle school.
I think it's mostly that they're more difficult to persuade that their delusions aren't simply them being smarter than the normies around them. That trait isn't unique to high IQ people or unique or people whose choice of reading material and social scene is based around the idea of accessing advanced modes of reasoning, but it likely is more common amongst them
> The smarter you are the more elaborate a prison you can construct for your own brain.
Amazing one-sentence summary for so much of what is going wrong right now.
Brilliant, rich people trapped inside their own mind palaces, spending vast sums on remaking the world into something that matches their interior delusions.
> Sometimes I think one of the evolutionary counter pressures that has likely prevented human IQ from being driven entirely up and to the right is that high IQ increases propensity for delusional thinking.
I had often had this thought too. Imagine if humans were 100x more intelligent, would we have necessarily survived for 200k odd years? Society, technology, morality all evolve at different paces and if one “outcompetes” the others then an imbalance can happen and humanity could be wiped out.
Imagine if a caveman was super intelligent enough to build some type of weapon of mass destruction but humanity doesn’t yet have enough experience to police this person or have laws or power structures to keep themselves safe. If you consider language and “memetics” technology than you have to consider beings 100x smarter could also be master manipulators capable of leading the group off the ledge. so many ways civilization could be wiped out.
Really the only reason we are around today is that society as a whole’s ability to implement strategies at recognizing dangerous groups and individuals outpaces the individual or small group’s ability to wield technology for dangerous means (well at least up until now).
