> I think the reason I find this so upsetting is that, despite the risk of bots, I like to engage in discussions on the internet with people in good faith. The idea that my opinion on an issue could have been influenced by a fake personal anecdote invented by a research bot is abhorrent to me.
I like Simon's musings in general, but are we not way past this point already? It is completely and totally inevitable that if you try to engage in discussions on the internet, you will be influenced by fake personal anecdotes invented by LLMs. The only difference here is they eventually disclosed it, but aren't various state and political actors already doing this in spades, undisclosed?
I keep seeing this take, and it makes me mad. "The house is on fire, didn't you expect people to start burning to death? People will inevitably die, why discuss when it happens?"
Engineering is fundamentally about exercising the power of intelligence to change something in the physical world. Posts to the effect of "<bad thing> is inevitable and unstoppable, so it isn't worth talking about" strike me as the opposite of the hacker ethos!
I think the other thing to keep discussing is that doing research, or otherwise using an LLM, to manipulate people's emotions without disclosure, is unethical.
By the way, people die in house fires from toxic smoke inhalation and a lack of oxygen. Engineers created smoke detectors and other devices to lower the risk of fire due to electrical shorts, gas leaks, etc., and to create fire suppression systems.
People still die because they didn't replace batteries, didn't follow electrical cord/device warnings, or left candles or other heat sources unattended. We discuss these events as warnings and reminders that accidents kill when warnings are not followed, when inattentiveness allows failure to propagate, and as a reminder that rarely occurring events still kill innocent people.
Maybe this will motivate people to meet in person, until that is also corrupted with cyber brain augmentation and in-person propaganda actors, rather than relying on only online anecdotes.
With online media, meetings in person are still corrupted by their skewed view from online sources. Such physical meetings would likely end up reinforcing the corruption!
I see this as further discounting the importance of anecdotes and personal experiences when making decisions that affect populations.
Yes, we know that personal stories can be compelling, and communicating with someone with different experiences from ours can be enlightening. Still, before applying these learnings to larger groups, we should remember that individual experiences do not capture the entire population.
I like Simon's musings in general, but are we not way past this point already? It is completely and totally inevitable that if you try to engage in discussions on the internet, you will be influenced by fake personal anecdotes invented by LLMs. The only difference here is they eventually disclosed it, but aren't various state and political actors already doing this in spades, undisclosed?