You don't need to use these tools if you don't find it worth it. If you want to write the same sort of JS as we wrote in the old days you can still do that. If you want to write modern JS, you'll either need to restrict your audience to the most recent browsers, or you will need tooling to help you.
But it's perfectly possible to write a modern PWA using modern js, modules, and a library like react (loaded from a cdn) without any build step at all. I've done it, and it's not all that hard as long as you don't mind only targeting the latest browsers (and writing out a bunch of file names manually).
Honestly looking forward for WebAssembly to mature, just seeing what Microsoft has done with Blazor[0] is impressive, and that's just scratching the surface.
But it's perfectly possible to write a modern PWA using modern js, modules, and a library like react (loaded from a cdn) without any build step at all. I've done it, and it's not all that hard as long as you don't mind only targeting the latest browsers (and writing out a bunch of file names manually).