If you rewind 10 years to a time before waves of big tech layoffs and the market for engineers was very much in favor, you'll find your answer. The explosive growth in hiring coupled with FOMO-driven risk taking resulted in many projects which were essentially resumeware for engineers. This environment permitted people to escape strategic thinking and any consequences.
Imagine having to deal with multiple data stores for your daily development but then go on-call for a week and have to deal with twice as many plus the inevitable lack of runbooks etc.? I personally have a big tech experience with the same data store proliferation in an organization with pretty solid RDBMS use. In the last 3-4 years we've been undoing the damage and migrating data. Toil for engineers is lower and uptimes are better.
At the time there were _no_ tools that scaled to what people needed so they had to build their own.
It's easy to look back and think we were idiots for doing it that way but you need to remember that the average server in 2014 has the power of the average smartphone today.
Imagine having to deal with multiple data stores for your daily development but then go on-call for a week and have to deal with twice as many plus the inevitable lack of runbooks etc.? I personally have a big tech experience with the same data store proliferation in an organization with pretty solid RDBMS use. In the last 3-4 years we've been undoing the damage and migrating data. Toil for engineers is lower and uptimes are better.