I see Make It Fast as a discrete step after Make It Right, because if it's made Right, then you can swap out the parts where optimizations need to be made. While you can't always know where slowness will occur, you do know that it will happen one day, and design the system so that swapping the "slow part" for a "fast part" is relatively trivial. That's the Make It Right part.
An well-designed API should behave so that you can fudge up whatever is happening behind the scenes, so long as the outputs remain the same based on the inputs. Design for consistency and idempotency. Implementation (read: behind the scenes) details are just that, details, and subject to change. If your implementation is tied to your interface, there are bigger problems and you skipped the Make It Right part.
Most of "make it fast" in modern software is fundamentally architectural in nature. If your architecture is not designed for performance or efficiency then no amount of module swapping will make it fast in any kind of absolute sense. And swapping architectures is tantamount to a rewrite.
Most performance has to be intentionally designed in from the beginning if it matters.
I will say, I don't think I've ever faced a performance issue that was caused by poor architecture. Maybe I've been exceedingly lucky. But pretty much all of the performance issues I've encountered that I can remember are in individual queries or functions / methods (or sometimes, a group of functions / methods); discrete units of code that could be tested, changed, and fixed without any sort of re-architecture or major rewrite. Or maybe we're using different definitions of "architecture" here.
My exact case was something like this: I've made a bunch of style libraries for web (my latest is Tamagui). The one before Tamagui was similar, it had variants and a `styled` helper, but it didn't output to "atomic CSS". A full year plus into the development of it it was working alright but was quite slow due to all the crazy CSS is was generating and inserting all the time. Atomic CSS really helps this in many ways.
So I dove in, technically I felt I could keep the API surface the same. But after about a full month of refactoring it to work with atomic CSS I found many problems. There are just some fundamental limitations to the API design you must enforce to make it work, and without such you really can't merge things properly. It's hard to explain without writing a mini-book, but needless to say the API surface very much can dictate the performance, and if you stuff your API with a bunch of features before making things fast, you may end up like me having to basically start from scratch.
My take: work/right/fast is a loop you must run many times. They also bleed into each other. Sometimes you do work/right and it feels fast, but you haven't deployed it at scale, so you never realize it's not fast. Keep your API as simple as you can, try and hit the fast part somewhat early before you add many features, and don't be afraid to take all your lessons and restart things. If fast is important to your lib, making it right must also be done in tandem with make it fast.
An well-designed API should behave so that you can fudge up whatever is happening behind the scenes, so long as the outputs remain the same based on the inputs. Design for consistency and idempotency. Implementation (read: behind the scenes) details are just that, details, and subject to change. If your implementation is tied to your interface, there are bigger problems and you skipped the Make It Right part.