In my experience LLMs offer two advantages over private thinking:
1) They have access to a vast array of extremely well indexed knowledge and can tell me about things that I'd never have found before.
2) They are able to respond instantly and engagingly, while working on any topic, which helps fight fatigue, at least for me. I do not know how universal this effect is, but using them often means that I can focus for longer. I can also make them do drudgery, like refactoring 500 functions in mostly the same way that is just a little bit too complicated for deterministic tools to do, which also helps with fatigue.
Ideally, they'd also give you a more unique perspective or push-back when appropriate, but they are yes-men too much right now for that to be the case.
Lastly, I am not arguing to not do private thinking too. My argument is that LLM-involved thinking is useful as its own thing.
Re: "yes men" - critical thinking always helps. I kind of treat their responses like a random written down shower thought - malicious without scrutiny. Same with anything that you haven't gone over properly, really.
The advantages that you listed make them worth it.
The output of the prompts always needs peer review, scrutiny. The longer is the context, the longer it will deviate, like if a magnet were put nearer and nearer to a navigation compass.
This is not new, as LLMs root are statistics, data compression with losses, It is statistically indexed data with text interface.
The problem is someones are selling to people this as the artificial intelligence they watched at movies, and they are doing it deliberately, calling hallucinations to errors, calling thinking to keywords, and so on.
There is a price to pay by the society for those fast queries when people do not verify such outputs/responses, and, unfortunately, people is not doing it.
I mean, it is difficult to say. When I hear some governments are thinking in to use LLMs within the administrations I get really concerned, as I know those outputs/responses/actions will nor be revised nor questioned.
1) They have access to a vast array of extremely well indexed knowledge and can tell me about things that I'd never have found before.
2) They are able to respond instantly and engagingly, while working on any topic, which helps fight fatigue, at least for me. I do not know how universal this effect is, but using them often means that I can focus for longer. I can also make them do drudgery, like refactoring 500 functions in mostly the same way that is just a little bit too complicated for deterministic tools to do, which also helps with fatigue.
Ideally, they'd also give you a more unique perspective or push-back when appropriate, but they are yes-men too much right now for that to be the case.
Lastly, I am not arguing to not do private thinking too. My argument is that LLM-involved thinking is useful as its own thing.