I will answer you with a lengthy anecdote. A couple of years ago I wanted to buy something from a popular online shop. The shop offered various payment options. I won't bother with details, so let's make it simple: option A was no risk on their side but time consuming, while B was higher risk but fast procedure. I was time pressed, so I was happy to see B in their FAQ. After signup and checkout there was no option B for payment. I thought it was a bug in their site and sent them an email.
I then decided to have it shipped directly to my parents (I wanted to use the item over the holidays) and set up an account in their name. After checkout, option B was suddenly available. Yay, they fixed it. I made the order.
Back to my account: option B was still missing. Huh.
I then made several new accounts using various addresses over the city. The pattern that emerged was that in more affluent zip codes option B was available, in poorer parts it wasn't.
Several days later, my email was answered: they claimed not to offer B to any new accounts on principle. This clearly wasn't the case.
My sample set wasn't big, and maybe it was a bug or A/B testing plus coincidence, but it made me think.
Big brother will not kill you or bite you with big fanfare. Instead, you will be struck by inconveniences. Online shopping will be more pleasant for some and maybe even impossible for others. That flashy item you want will be available for others but you will be told that it's out of stock. All of this will be explained away as mistakes.
What? It sounds like the problem is they didn't have enough information about you.
If they had more info, they could avoid blanket policies such as avoiding riskier shipping options with all new customers, and use with customers they can prove are not fraudulent.
But here's my anecdote:
I bought a metal collar extender at Macy's with my credit card. It was an impulse buy at checkout and I have never searched, emailed, liked, etc. this product in my life. I get on Amazon a week later and I have a recommendation carousel of metal collar extenders. I was so surprised I even checked my email for a receipt to see if that's how Amazon retargeted me [there was none].
But then I just stopped caring, because I realized that if anything Amazon was incurring an opportunity cost for not recommending me something more useful, and I have more important things to do with my time than to worry about how I'm being retargeted on Amazon.
> What? It sounds like the problem is they didn't have enough information about you.
That is part of my point. It will never be perfect, but perfect enough to get a benefit for businesses. If this leaves you as part of a small percentile at the road side, that is your problem not their's. In the end, you may be regularly unfairly disadvantaged because of it nevertheless.
This is an example of a Nirvana fallacy - where you are saying it should be abolished because the "perfect solution for everything" is unattainable. If anything, your anecdote proves that should be made even better, not abolished.
Besides, the business lost a customer with you, so it's obviously not perfect enough for them either.
It is incredibly easy to see where your anecdote could go wrong. What if you had bought something much more personal or embarrassing?
Condoms? Plan B? Dildos? Lube? Imagine having to explain to a family member sitting next to you why those items are showing up in your Amazon recommendations.
I recall an example a few years ago about a store's marketing system telling a father that his teen daughter was pregnant before she ever told him. Because the store's system had correctly inferred from her previous purchases that she was pregnant and started giving recommendations for stuff like diapers.
Suppose a homosexual in the closet was buying sexually explicit things or an atheist in a religiously strict home or country was buying blasphemous materials.
Ironically, your anecdote presents much worse scenarios than the parent comment's anecdote.
Slipping down the slope of embarrassing adult toys shows a lack of understanding of how these networks work.
Again, this isn't a problem with retargeting. What if I just bought sex toys directly on Amazon and they made product recommendations for it while I was browsing at work? Or what if Facebook and Google showed me adult ads while reading Forbes.com?
That's on Amazon et al for displaying adult sex toy recommendations. Same as your nightmare scenario of where I'm a persecuted homosexual in Saudi Arabia, and Amazon makes homosexual product recommendations to me while the religious police man is looking over my shoulder.
"That's on Amazon" but you're bearing the consequences. What consequences would there be in the nightmare scenario for Amazon? None. They get the power and not the responsibility.
They have your problem now and already address it. It isn't unique to retargeting.
They lose customers. Ad networks lose websites. Walled gardens lose users.
That's why you don't see promoted dildos on Amazon, Forbes, and Facebook even now - and why your nightmare scenario is just a bad daydream about rubber cocks.
I replaced a leaky kitchen faucet with one I bought online. For weeks afterwards, banner ads for that exact same faucet kept appearing everywhere. Did they think I had two kitchens?
I notice the targeted advertising all the time, and I've never been tempted to buy. Their predictive algorithms on what I'd like to buy next are hopelessly wrong and often (like the faucet) simply incompetent.
> Big brother will not kill you or bite you with big fanfare. Instead, you will be struck by inconveniences. Online shopping will be more pleasant for some and maybe even impossible for others. That flashy item you want will be available for others but you will be told that it's out of stock. All of this will be explained away as mistakes.
It is not enough for a cyberpunk action movie. The consequences for society are chilling, however. Picture a society in which everyone has even more reason to behave and live in a certain way just so they don't get misjudged by some algorithm.
Hasn't it always been like that? Just replace "some algorithm" with "social norms". I don't see a qualitative difference.
Wait, actually, one difference. The algorithm adjusts much quicker than social norms, which is why it seems to affect you more. Changing of social norms is usually only observed by people who are older than the average HN reader.
I had a similar thing just recently. I was sitting in a hackerspace trying to book a hotel. It wasn't possible there because they wanted a credit card (just as safety if I don't show up, I was going to pay in the hotel anyway).
Since I don't have a credit card, I IM'd a friend if I could use his credit card for the booking. I went over to his place, and when I did the booking process at his home, suddenly I could book without a credit card.
My hypothesis is that the hotel website didn't trust me in the hackerspace since I was connecting to them over a VPN.
That has everything to do with data collections. It won't always be something obvious as poor zip codes. When data of many people is collected it enables the establishment of certain behavioural types. Then when you come along with the few data points they know of you, the gaps can be filled. What goods you buy, travelling behaviour, waking hours can all be used to characterize you. Regardless of whether this labelling is fair.
In the U.S., it is an illegal practice in the context of lending.
It's also leads to really ridiculous situations. For example, living in Brooklyn, NY made it impossible to rent a car in the 90s without an existing relationship with a rental agency. Living in a black neighborhood made it impossible to get a mortgage.
I'm aware that we have a multitude of (often confused and nonsensical) laws, which in some cases force businesses to ignore data and make bad decisions.
My question; what argument can you make in favor of enacting such laws if they didn't exist? And if you make such an argument, does it also imply that small females should ignore the fact that Delhi is the rape capital of India? Or that pizza shops should put their drivers in danger of robbery?
In NYC, a decision was made and actions taken to improve the safety and security of the environment so that a woman can safely walk the streets unmolested.
I then decided to have it shipped directly to my parents (I wanted to use the item over the holidays) and set up an account in their name. After checkout, option B was suddenly available. Yay, they fixed it. I made the order.
Back to my account: option B was still missing. Huh.
I then made several new accounts using various addresses over the city. The pattern that emerged was that in more affluent zip codes option B was available, in poorer parts it wasn't.
Several days later, my email was answered: they claimed not to offer B to any new accounts on principle. This clearly wasn't the case.
My sample set wasn't big, and maybe it was a bug or A/B testing plus coincidence, but it made me think.
Big brother will not kill you or bite you with big fanfare. Instead, you will be struck by inconveniences. Online shopping will be more pleasant for some and maybe even impossible for others. That flashy item you want will be available for others but you will be told that it's out of stock. All of this will be explained away as mistakes.