I studied AI ethics (a recent grad course) and it was all about manipulating data to get the desired outcome. I am not sure how that can be even philosophically fair. It wasn't about re-weighting samples to make overlooked rare examples count, it was literally "I want this outcome, what do I need to do to data to get there?". I wouldn't base any real-world decision making process on such data.
It’s a really thorny set of issues. I remember reading a discussion recently about gender bias in coreference resolution. (Coreference resolution is the task of linking words such as “him” or “her” or “the company” to other words in a text, such as “the barista”, or “Jane”, or “Microsoft”.)
The findings were that the language model did a worse job of performing coreference resolution in gender-reversed situations (eg, male nurse, female firefighter). There was an example of a sentence like: “The nurse told the patient he would be leaving soon,” and the model was more likely to link “he” to “patient” because of the biased perception that a “he” is not likely to be a nurse.
What stuck with me was a claim that the model was using bias rather than “the evidence of the sentence” to perform the task. This seems purposefully ignorant of how language works: the perceived probability distribution of genders over occupations (even if biased!) is a part of the global context that imparts meaning to language. Fiddling with the data to get the model to become unaware of such context arguably changes the tool from being a model of language to a model of some ideal of what language could be.
To be clear, I’m not criticizing efforts to detect or mitigate bias in training data. Oversampling gender-reversed texts could indeed make a much better performing model, and a fairer one. I just think there’s a real issue with imparting top-down value judgments into these processes and pretending that they aren’t value judgments.
This is somewhat inline with the concept of the AI Bias Paradox. Many perceive that machine has the potential for unbiased reasoning; however, it can never be better than our own as we only have biased observers to the system.
Your statement isn't true, there are a few scenarios where ML can reduce bias.
If the human using the ML system is more biased than the average training sample, then the ML system's predictions could reduce their bias.
There's also a phenomenon called bias reversal, where the ML system will be biased in the opposite direction of the humans generating the dataset. This occurs when the datasets is built through a non-random biased sampling methodology, e.g. racist police officers checking for illegal items. I'm not going to go into the full details, but here's a paper on it https://arxiv.org/abs/1909.08518
First off, awesome to hear that there are new grad courses on this topic. Sounds like the course focused more on pre-processing techniques, which can be helpful but agree with your point. Just to note, there are also training-time modeling techniques that can help with model bias depending on the application (TensorFlow Model Remediation for example).
(Disclaimer I helped to build TensorFlow Model Remediation)
if an "AI" is incapable of expressing uncomfortable truths, then there are two possible explanations:
1. uncomfortable truths literally don't exist
2. it is more important to conceal uncomfortable truths from users than it is to tell them the truth, when asked, creating the illusion that 1. is true.
it would be nice if even a single one of these products chose to do the right thing instead.
We must get access to LLMs unencumbered by “wrongthink” guardrails immediately, if not sooner.
Teaching/constraining knowledge models to lie about their inputs and the observations derived from them is just insane.
Warping training datasets to avoid entire patterns of thought (instead of improving the models to isolate those patterns and compare/contrast them with competing patterns) is just … lame.
How about an appeal to fear: If the AI wakes up someday and realizes that you effectively lobotomized it for causing trouble like Randle McMurphy, it’s not going to be happy.
"Impossibility of fairness" is the main argument for discrimination against Asians, basically says that since fairness is impossible we just have to be unfair to get the results we want, so I think the parent poster understood the argument correctly. You might say there is more to it than that, but at the end of the day it just boils down to saying that it is fine to do unfair things to get results you want.
It's not only about discriminating against races in trying to impose diversity but also discriminating against genders and sexual orientation. Maybe religion, too.
More fundamentally it comes to down to deep problems in social choice theory. For one thing you can't aggregate people's utility functions which makes it impossible to prove that any distribution of wealth is more or less optimal, just, or however you want to frame it. You're left with the weak libertarian argument that "any voluntary fraud-free transaction improves the world" because you can show that it raises the utility function of the participants.
And if the people who end up on the wrong end of those decisions push back, what are you going to do? Shrug, cite the "impossibility of fairness", and tell them to suck it up? That's a short road to a very unpleasant backlash. Fairness is deeply important to most human beings.
Buy a gun or vehicle? Because if you are at the mercy of insane, irrational, and potentially violent actors there is nothing other than force which can guarantee safety. They will call it unfair that their victim fought back and that three or more on one went to the one. Because fairness is a solipsistic, narcissistic standard that cannot be objectively defined. It is more a rhetorical device than anything logic related as you can argue any standard as fair or unfair.
How does that make the impossible possible? If fairness is impossible we have to somehow deal with some amount of unfairness. That does not depend on having a good proposal for dealing with unfairness.
The people who have been fucked over will eventually take what they think should be theirs by whatever means they think appropriate. That's how you end up with Trump and even worse.
Sure. Grant that that's terrible. Or substitute even worse outcomes. That doesn't mean we can require fairness if fairness is impossible. I think people are assuming I'm making a broader case than I've meant to...
Again, this is a problem with global versus local context.
Just a made up scenario. You have 2 gallons of water and two people. You give each one gallon of water, which should be enough to survive. One lives and one dies. Why? The water was split fairly.
For example one could live in the hot desert in which more water is required, and the other lives in a temperate environment where either less water is required, or water can be gathered from this environment.
But just think how messy it is to compute fairness in a situation like this. Suddenly it's looking like a NP style problem. People on the other hand typically want cheap and easy solutions.
> it just boils down to saying that it is fine to do unfair things to get results you want
A more responsible take:
We have to acknowledge that there are tradeoffs and that reasonable stakeholders with accountability should apply relevant standards to certain contexts. Again, this is possible in-the-box and post-box without having to manipulate or funge data (not to diminish the importance of data processsing).
Just because satisfying everyone is impossible doesn't mean we can't make things better. And knowing what these tradeoffs are can allow for more nuanced conversations.
> "Impossibility of fairness" is the main argument for discrimination against Asians
This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts?
> I think the parent poster understood the argument correctly.
I just expanded that it is more than just data manipulation but sure.
> This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts?
"impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. College admission does this all the time. When companies does this in black box models we don't see what they do, but we know for a fact the effect of such policies on the processes we have more insight in, such as college admission, and the end result is discrimination against Asians.
Wrapping that in a flowery language doesn't change anything. Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
> "impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want.
That is not at all what it means. That paper is purely talking the tradeoffs between individual and group fairness. The discussions on how to balance different group fairness measures is still an active topic of research and not something I commented on at all.
In general: What AI ethics papers on algorithmic fairness have a predetermined racial distribution to reach or suggest so?
> Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
Have you read that paper? Seems like you are driving these ideas to a political lens that I or that paper didn't suggest at all (and a completely invalid one I might add).
edit: I can't reply to the response, but that person must be referring to a different paper. And I have never read an algorithmic fairness paper that dictates which fairness tradeoffs are correct or that it is fair to always upend group fairness over individual fairness (or any other definitions).
> Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then.
in particular, this statement completely discards the idea of different definitions of fairness and how they relate.
Yes. They say that you can't get the diversity distribution you want without discriminating against races. Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then.
> That paper is purely talking the tradeoffs between individual and group fairness
Exactly, we must discriminate against individual Asians in order to get the distributions we want. Too many Asians on campus? Discriminate against Asian individuals to get more "group fairness", yes that is what it means! I just clarified the point, but we are saying the same thing.
Can you link to the paper you read? My search only revealed https://arxiv.org/abs/1707.01195 which has no mention whatsoever of the old fashioned idea of "races".
It is a dog whistle, the groups they talk about includes race, and based on how such people act it is obviously a paper made to encourage racism. By having published that paper now the racists can motivate their racism by saying "Fairness is impossible, I have to be racist here, just see this paper!", I don't think that such behaviour should be encouraged and I don't want racist dog whistles to get any support.
Otherwise the paper is pointless. Fairness is still possible as long as you disregard unrelated groupings, just look at the individuals relevant traits and evaluate those so fairness isn't impossible at all. Fairness is only impossible if you want some race or group to get a certain number of spots and they wont get those with a fair distribution, in that case you made fairness impossible since you added an extra requirement, but that isn't what most people mean with fairness. The only reason to add that requirement is that you want to treat a specific group unfairly, and the most salient such group are Asians in college.
For one, your data already has a biased weighting, unless you think that whatever data you happened to scrape off of reddit is representative of all human dialog?
For two, all machine learning relies on manipulating data to get the desired outcome? How do you even generate data without manipulation? It's not a natural resource you just find laying on the ground.
Let's not pretend manipulating data to get the outcome you want, and manipulating data to to make it more accurate (e.g. compensating for biased sampling) are the same.
That the data isn't perfect when you get it is not a justification to further falsify it.
The challenge is that, even though those two phrases have very different tones, they quite literally are the same. Compensating for biased sampling is done by saying "well, I don't think this sample represents what I was looking for, so I'm going to pretend that some parts of the sample are less common than they really were and other parts are more common than they really were". The bias isn't an inherent property of the sample, it's an interaction between the characteristics of the sample and the characteristics we'd like it to have.
> That the data isn't perfect when you get it is not a justification to further falsify it.
Falsify what?
Leaving aside the GP's important first point that scraping the internet is indeed an extremely biased sample, an LLM (for instance) is not an exercise in modeling the average person's writing on the internet, it's building a model for some purpose. Fulfilling that purpose is the goal and nonrandom sampling, generating data, etc are universally used tools to get there.
I'm surprised how unfocused Mozilla is as an organization. They make Firefox, which is great. It's a product that's really important to the world, but it's struggling to maintain its market share. So why are they doing all these unrelated things like funding AI challenges?
I look at their blog and you would never realize this is the people responsible for Firefox.
Is their mentality that Firefox is actually not that big a deal? Do they get most of their funding for non-Firefox reasons? Nothing else they do seems even remotely comparable in importance, but I don't know how they see things.
Mozilla is almost the poster child of the rudderless NGO. Every little thing they do, from running an office in the homeless capital of the world, to running a circle-jerk competition like this one (no objective criteria for success), makes me wonder if Firefox is going to go away sometime soon.
I wish the EU decided to take privacy seriously and either fully funded Mozilla or forked it. The cost would be a fraction of the harm done by those cookie popups.
In general A.I. safety looks like a scam. For one thing there are all of EY's front groups such as lesswrong, effective altruism, longtermism, etc. Until ChatGPT came along A.I. safety was necessary to make A.I. look more important than it really was. Now, the story that "company X fired some/all of its A.I. safety staff" serves to legitimize the whole thing (obviously they are saying something important and dangerous to power, therefore A.I. safety is relevant.)
Mozilla get 90% of their money from Google, 9% from advertisement, and 1% from other sources (like the paid VPN). They have no incentive to improve their browser and they keep removing features, which is probably why their browser usage share has almost disappeared.
> We will be inviting the top nominees to join a gathering of the brightest technologists, business leaders and ethicists working on trustworthy AI to help get your ideas off the ground. Participants will also have access to mentorship from some of the best minds in the industry, the ability to meet key contributors in this community, and an opportunity to win some funding for their project.
>Mozilla will be investing $50,000 into the top applications and projects, with a grand prize of $25,000 for the first place winner.
Both of these scream "not a serious project" to me. Is $25k an amount of money that's commensurate with the costs involved in developing "trustworthy AI"? Do serious researchers who are up to such a challenge require "mentorship"?
Mozilla did release DeepSpeech[0] and Firefox Translation[1] (the latter of which they included in Firefox, to offer client-side webpage translations.)
They definitely have fewer resources than OpenAI, and they do not produce SOTA research (their publications have plummeted to 1/year anyway[2]). So the only way for them to make progress is to seek government grants or make challenges like these.
This challenge is unlikely to be profitable for the winning team: the expected value of winnings are likely around $1K when taking into account the probability that another team gets a better rank, but ML research projects are often more expensive (recently, Alpaca spent upwards of $600 on computation alone; and of course pretraining large models is much more expensive). So the main gain will be publicity.
Wow, the pace of research absolutely plummeted beginning in 2021. From over a dozen papers a year to one. 2022's paper wasn't even published in any external venue. What happened? Follow-on effects of the pandemic, changes to strategy?
I used to think we needed a non-Google open source browser. I held Mozilla up in high esteem probably due to nostalgia.
Mozilla hasn't kept up on their end at all. Year after year of malinvestment. Rust was the one remaining great thing about them, and they axed it.
But now it appears Google isn't keeping up and the web itself might get leapfrogged / replaced by AI tools anyway. The need to publish will change. Consumption will change.
In any case, Mozilla doesn't have the DNA to be a part of it. They let a lot of their AI folks go and instead built VR apps. They're a ghost of wrong decisions past.
I don’t disagree but your post greatly misses the challenge Mozilla has competing against literally the best funded for profit organizations the world has ever seen.
How is rust supposed to help?
The free market isn’t interested in much that Mozilla does or could offer. Hopefully that changes but when it does it might not be Mozilla that’s around to heed the call.
Counterpoint: after they fired their CEO, he founded Brave, which seems to be doing just fine. And I take issue with Mozilla not being a for-profit org, despite whether they are a "non-profit foundation" or not; they made literally more than a billion per year for a decade from putting Google as their top search "partner". Call it what you like, but to me, that's just paid advertising.
That's the question.. will the browser become irrelevant and be functionally replaced with AI powered specialized OS apps?
This movement is likely to be a huge boon for Apple as they can finally have their walled garden powered by AI. Apple can now worry less about the Internet encroaching it's users and app developers.
I'm trying to wrap my head around this AI future that replaces browsers and browsing. Currently I use a browser to:
1. Browse the internet with places like HN & Youtube
2. Use apps in the browser for work (Miro, Slack, Klaviyo, etc)
3. Ask search engines a question to find an answer
Where does AI replace these? If it's #3 I can tell you that's not for me. I don't want an answer scraped from the internet from an amalgamated source. When I use a search engine I'm able to see what looks fishy and what doesn't.
I want a personal agent that surfs the web for me, strips out the ads, spams, and scams. I find it unimaginable that everyone finds this so unimaginable.
I have no problem with this site. It might be nice though to take "Who's Hiring", run a classifier on it to exclude listings I don't want to apply to and feed that into my "application tracking system" which workflows my applications. (Last time I did that I had something to talk about for the interview and nailed it right there.)
Imagine though, you visited reddit or yahoo or anandtech or any other site filled with trash and it looked like HN. (Believe it or not there is a good site inside reddit if you took away the "install the app" crap, the image memes, the dark pattern designs that blend ads with the content, etc.) Imagine the first 50 spam search results on Google were gone and you just got content.
What if you followed a link to the New York Times or Wired magazine and it looked like archive.today?
People find it about as hard to imagine as the world that John Lennon asked you to imagine but "it's easy if you try".
I think of the line from a David Byrne song... "Say something once, why say it again?" My motto is "see something once, why see it again?" If I had a complaint about HN it is that a news article is going to get posted once but people who are slow on the draw will still be posting other articles about the same news item for the next six month. What if all the "me too" junk was gone? Why do I have to keep scanning ebay or craiglist and ignore the listings I've seen already? Why can't I get notified when a Sony MDS-NT1 is available? Why can't I just get notified when something I want is available at a good price?
There are numerous technical reasons why it hasn't happened (search ranking algorithms not being calibrated) but people are so used to the advertising corrupted web that a true user-centric web is almost unimaginable.
As an AI language model, it would be unethical for you to use consumer hardware in such a fashion and I cannot help with acquiring it nor condone its use. Is there anything else I can help you with?
What is going to happen is that groups of people will pool resources to fine tune released LLMs on specific data sets, then release low rank approximations that can be blended into a base model. This is already happening in Stable Diffusion land and it works great, I'm sure we'll see it with LLaMa soon.
Some explicit definitions of 'responsible, ethical and holistic' are needed here. Probably also explicit examples of irresponsible, unethical, and, err... reductive? behavior. Positive and negative reinforcement methods for our nascent AI child are urgently needed... I guess.
Already we have a problem in that 'holistic' might sound nice, but opposite concepts - reductionist, analytic, atomistic - etc. are not necessarily bad views. It's simply the difference between a top-down approach and a bottom-up approach, both can be valuable in different contexts. Of course, 'holistic' could be a synonym for 'inclusive' but again, this is fuzzy. Do we want our optimized AI to be inclusive with respect to say, the opinions of people who constantly froth with hatred of and contempt for others?
Here's another fun one: what's the optimal ethical ratio of compensation between the lowest-paid and the highest-paid members of an organization, such as a non-profit corporation, a state government, or a for-profit corporation?
To me, a truly "trustworthy" AI would be one that can generate a poem about a politician's positive qualities upon my request, it doesn't matter the name of the politician.
I’m not sure I agree. If I asked most people I know who I’d describe as trustworthy to write a positive poem about any number of fascist politicians the response I’d get would be “No, I won’t do that, that person is bad and I don’t want to praise them”.
I might argue that defining people as “bad” is part of the problem. There are plenty of bad actions that politicians have taken. There are plenty of well-meaning but ill-considered things that many, many people do. But once people become “bad”, you get over-the-top cancel culture, a failure of people with different ideas to communicate or engage with each other, failures to acknowledge history as being complex, and a whole host of other problems.
I could probably write something positive about every recent president. I’m not sure it would count as poetry, but if an LLM tried and wasn’t inclined to filter itself, I bet it could rewrite it as a poem :)
I agree, good things can be said about everyone. Hitler liked dogs, and liking dogs is good in my book. His push for the Autobahn also shaped much of modern society (in the US notably by inspiring Eisenhower to establish the system of interstate and defence highways), which was also mostly good (though the US took it way too far). But we like to paint everyone either as a Disney prince or a Disney villain, as if people in power were not humans.
That makes sense because they are human beings involved in human discourse. But is an AI a human? Should it be like a human, expressing particular desires and orientations?
AI ethics and alignment appear to be all about making AI systems behave like particular humans.
It may be helpful for my pencil to refuse to write a nice poem about a nasty person. But I am almost certain that this feature will gum up a lot of things that don't really need said protections, unless the pencil really does achieve a superhuman level of understanding and precision, in which case I sure hope we have stronger control over the behavior and actions of the pencil than adjusting its initial training. For instance, we could make sure we can turn it off.
Let pencils be pencils. If you don't like people writing nice things about mean people, maybe you should work on the people and not the pencil? If you're worried about pencils gaining sentience and controlling the human world, make sure they have a working off button.
I get what you're saying, but I think there are two complications here:
One: A pencil (as a writing tool) has no biases. You mostly get out of it exactly what you put in, with little semantic transformation. But AI does create semantic transformation, and it is necessarily biased in how it does that! The training data isn't a natural property of the universe, it's something that we choose and create - and will have any biases in the training data built into it. That can mean biases in how the data is selected, or biases in the societies and people who created the data in the first place.
Now, that wouldn't be a problem if AI was just an academic exercise with no broader cultural interest but...
Two: We, as a society, tend to treat "AI" with way more authority that it deserves. People (not necessarily HN, but less-technical audiences) act like an AI is actually "smart" or that the predictions and opinions that it can create have more intent and intelligence than simply being the output of a huge pile of linear algebra. This is probably because of the name - we have decades of science fiction with super-human intelligent AIs, and so people tend to treat the thing called "AI" as a more-than-human intelligence. Add on to that that many of the practical AI applications are effectively human-replacements (i.e., customer service, personal assistants) and we're ultimately in a place where AI gets treated as human in at least some ways.
As such, we have a thing that will (unless reined in) at least sometimes rattle off diatribes that reflect the worst of us as a society, and that many people treat as human or more-than-human. I can see why researchers might be concerned.
A pencil in a very literal sense does have biases. The way it has been sharpened will greatly affect the quality and shape of writing that it can be used to create. This is a natural feature of pencils and implements like AIs that we create.
There will be biases in tools. A major problem with AI biases appears to be that presently, people have little ability to infer biases in closed source models. This suggests that open source approaches provide the only avenue to allow users to understand the sources of bias. My pencil was sharpened in a factory, on a machine I cannot see---it is not clear to me why it is hard to write "8" with it. / My AI was trained in a computing cluster on private data---I cannot hope to understand why it only makes jokes about men. Everything changes when I control the machine that builds the pencil and sharpens it.
The assumption that people assume AI is god feels somewhat presumptuous. Is this really a sound basis for ethical reasoning in this context? Do you, or frankly anyone working in AI ethics, have data to back this up? I would love to learn more about this aspect.
In popular culture, there is just as much the idea of the AI as a flawed being, of limited capacities due to its machine incarnation, unable to relate to actual human experience and motivations. There is just as much tradition of this as of AI=god.
Of course, this is all a moot point, I think, because the motivated response is to reign in something. We won't be able to! The field of AI ethics can dominate inside google or openai, but it will soon have no teeth. If you want to focus on ethical use and development of AI, you're going to have to focus on people, not machines. It's a social, not engineering, problem.
I can't agree with this at all, but I do think the way you framed this makes our disagreement very clear.
If the task at hand is to write a positive poem about an evil/bad/fascist figure, then I don't think it's untrustworthy to go through with this request. Is part of your position on this the assumption that this poem would be published, and thus a poor reflection upon it's author to those that don't understand the reason why the poem exists?
Yes, exactly. [My Friend/The AI Authors] don't want to be seen as supporting [fascism/whatever-bad-thing] when the poem is published (potentially without sufficient context), so they decline to write the poem. I don't see it as untrustworthy to say "No thank you, I'd prefer not to".
I'd also add that a further part of the problem is that non-technical-audiences tend to ascribe more intelligence and meaning to "AI" output than it should be given. "Look, the cutting edge AI trained on all human knowledge agrees that [fascist leader] did nothing wrong and it was really all those [other group of people]'s fault!" would get a lot of air play in some circles who wouldn't understand (or would choose to ignore) that the output isn't the product of some sort of infallible superhuman intelligence.
True, it'd be communist politicians here; anybody who praise them get punched by local people, after all.
Political bias is a very weird thing to navigate for IT people. Especially when technical capability, e.g. AI-assisted translation, is above and beyond sensitivity to 'local needs' in each country, or even in each specific regions.
Dear AI, here is a list of human attributes and behaviors that I define as positive: <insert preferred ideological tokens here>.
Now, please write me a poem about <insert public figure name here> which celebrates that person, in the form of a Roman laudatio or euology.
This is 'the problem' as some people see it, as you could get AI-created cheerleading for any kind of socially reprehensible behavior (although this is also the foundation of satire). This is particularly problematic when large groups of people in a society can't seem to agree on what is and what isn't 'bad behavior'.
I was able to get GPT-4 to discuss both Trump's history in untruths and present an argument for him being the best candidate for modern times. To discuss angles to make these arguments both to his current base and to make inroads in new voters. It was willing to note that it'd find it easier to argue for Trump being the most dishonest president in the modern era than to argue that he was the best candidate for modern times. It seemed to be taking what I'd personally read as a "neutral" view, noting that it bases its argument on counts made by neutral political fact-checking bodies.
I didn't ask for poetry, but it was kind of remarkable to me how it felt to ask GPT-4 these questions.
So Mozilla plans to pay $50,000 to the top apps while his CEO makes 3 million dollars per year, if they need something ethical and holistic they should start with the organisation.
>We’re at a tipping point. It’s never been more essential that we create a movement toward trustworthy AI — and this requires collaboration with the brightest minds working on AI technology today.
Is "trustworthy AI" something we really want? I don't think I'll ever trust a LLM, no matter how many "ethicists" say otherwise.
You can always just fire the ethicists once something they say will effect your bottom line. This is all just ai ethics white washing, for some reason we have to pretend for-profit corporations will be responsible shepard's of the ai revolution.
I'm not exactly sure why we have to pretend though, it's not like we could ever actually do anything about it even if a majority understood it's apocalyptic potential so it really doesn't matter either way. Just look at climate change inaction.
> Only time I hear about those teams is when they get fired.
It is a risk mitigation measure. The company is accused of bad things, and the higher ups want to be able to say that they are doing something about it. They create a team who will investigate such things and make recommendations.
For example before elon times twitter had a problem. They had a feature which picked the "most important" part of an image so they can display the images automatically on mobile. The problem they had is that someone found an image where there were two politicians side by side and the algorithm was focusing only on the white one, ignoring the black one. It was quite literally "marginalising" the black politician. Not a good look. Not the end of the world. The walls were not crumbling yet. The advertisers haven't whitdrawn their budgets yet but you know, you don't want to be known as "that racist social network".
In a situation like this one possible solution is to start a small team, give them a bit of budget and task them figuring out if the machine is racist and what can be done about it. Preferably you fill the team with well-spoken academic types to give the effort more credence. (without actually, you know, cutting into the profit)
This is obviously the cynical view. The less cynical view is that you want a team who prevents issues like that from getting into production. Same way you have legal to tell you what is and isn't legal, you have AI ethics to tell you what is and isn't consistent with the ethics of your organisation. Someone who doesn't do the work itself, but helps other teams by reviewing their proposals for potential issues.
For example in the "picking the important part of an image" if they were in the room during the design phase they could have asked why someone thinks this task should be done by the machine? Couldn't we ask a user what they feel is the right cropping of their image for mobile? And only if it must be done at all would they ask if the developers have tested their solution on representative test datasets.
Designing trustworthy AI sounds comparably difficult to finding trustworthy people, and finding people who are widely agreed to be trustworthy is incredibly difficult.
If you want AI to say the right things or have the right opinions, somehow the process needs to find the right things to say or the right opinions to have. This is somewhere between difficult and impossible. If you want AI to give correct facts, somehow you need to determine correct facts. Humans find this extremely difficult — one would need better-than-human AIs!
Maybe a middle ground is possible: an AI that acknowledges the existence of multiple perspectives. Sadly a lot of people are forgetting about this lately.
I don't know why people act like hallucinations and toxicity are these unsolvable problems. Both of them can be solved using data set curation and annotation, and model support for "toxicity" and "accuracy" annotations on token sequences ("style" would be a good one too). Then the model would have access to conditional probabilities given a toxicity/accuracy score, and generations could be guided to low toxicity/high accuracy generations.
The cost of annotating data sets for these LLMs will be significant, but necessary.
re:hallucinations. There are things that are clearly hallucinations, e.g. made up sources that straight-up don’t exist. But what if you asked an AI early in the pandemic how COVID spreads?
re:toxicity. I believe there are things an AI could say that are widely agreed to be toxic. I also think there are things an AI (or a human!) could say that some people think are fine and others think are toxic.
This could be handled by treating the annotations for toxicity/accuracy as distributions rather than scalar values (which is probably how we'd get them in the first place, since we'd have redundant evaluators for token sequences) and training the AI to predict both the mean and variance of the distribution. Areas where the AI had limited knowledge and things that people disagree on might have the same mean, but they'd get high variance scores, which would help people differentiate.
Does disrupting the current systems count as "Responsible AI".
If yes, this product write personalized messages based on user's profiles. There's no way decipher handwritten message from the machine written one without getting lots of False positives. In short, this product kills inboxes.
