It is worth mentioning that Eliezer Yudkowsky doesn't believe in the scientific method, and thinks it can be replaced with pure Bayesian reasoning, without, as far as I can tell, any data collection.
I'm not even sure I should bother replying to this. But, like many, I'd consider the scientific method properly applied to be a special case of probability theory, modulo some social rules that might technically depart from Bayesian reasoning but are supposedly there to create good equilibria among less than perfect reasoners. This involves observation, natch.
The parent commenter is presumably beyond all help, but anyone interested in an example of my views on this point may consult "A Bayesian view of scientific virtues" here: https://arbital.com/p/bayes_science_virtues/
I am amazed you took the time to reply to this as well, but, just so its on the record, having read more than a little of your material, I do consider your entire "rational" movement a bit of a cult.
The 'scientific method' is not something that can be reduced to formal reasoning. Bayesian inference is a type of formal reasoning for making predictions, and as such, is a mathematical 'toy model' that doesn't correspond to the real universe.
Yudkowsky worships pure mathematics, but he always had it 'arse backwards'. It's not pure mathematics that's the ideal, it's numerical and iterated methods and heuristics (cognition). Pure math can only be applied to idealized situations, whereas numerical methods and heuristics apply everywhere. So in fact, it's numerical methods that are fundamental, and pure math that's the imperfect idealization!
Yudkowsky read too much Tegmark in his youth and was sucked in by the idea that 'everything is mathematics' (or 'everything is information'). But to repeat, this is all 'arse backwards'. Thank goodness that I read some Sean Carroll and debated with a friend of Sean's on his forum; that's what finally talked me out of all that Tegmark multiverse/'reality is a simulation' nonsense.
It's the physical world that's fundamental, cognition is next level of abstraction up, and pure mathematics is a top-level abstraction (it's not fundamental). As Bayesian inference (and all formal methods) are part of the domain of pure math, they can't serve as a foundation for rationality.
Cognition is more fundamental than math (because it's closer to the base level of reality - physics).
As I commented recently to Scott Aaronson on his blog, what distinguishes cognition from pure math, is that pure math is about fixed equations, whereas cognition is about heuristics , iteration and numerical methods. But in fact, P≠NP implies that cognition is the more fundamental. See:
So for instance, AIXI (a much touted mathematical model of general intelligence), is the 'fake' (imperfect) solution, whereas a workable heuristic implementation would be the correct (perfect) one. This is the complete reverse of what Yudkowsky thinks.
I'd like to assure anyone reading this that the above commenter has no idea what Eliezer Yudkowsky thinks. I, with my considerably greater knowledge of that subject, can testify that among Yudkowsky's beliefs is "Bounded agents are more impressive than unbounded agents."
(In general, you would be very very wise not to believe someone who claims that I believe a thing, until you have seen the original text, in its original location, in full context, written by me under my own account, plainly and unambiguously saying that I believe that thing. Even then I've been known to change my mind later, as is a sane person's right. But most of what I'm wildly rumored to believe is more completely made up out of thin air than anything I've changed my mind about.)
