It's a cost/benefit thing. You should try to design your program sensibly when it comes to performance, because the cost of a failed architectural decision is high. What you shouldn't do is micro-optimizations which make the code longer and harder to read.

I'm not sure a 'readable' preemptive micro optimization is any more sensible. I think we should limit 'sensible' to preemptive macro and architectural optimization. A programmer should examine what their compiler is spitting out before they decide they can do better.

What is a "micro-optimization" is debatable, but if you have two ways of doing the same thing, and you can do the fastest one without impact safety or readability, it's not wrong to pick the fastest one (for instance, no point in building list A in order to append A to list B when you can directly add stuff in list B, like the code I refactored away one hour ago).

As for architectural optimization, I disagree. There are things you're really going to have a hard time adding post facto. If you know you're going to need performance, it's important to know what kind of program you're going to build (this may include benchmarking prototypes).