As an AI/ML researcher who publishes in this area regularly, I will be using this as a case study for AI ethics classes. That this is allowed to go on is shocking.
> In 2018, a programmer named “coolwulf” started a thread about a website he had made. Users just need to upload their X-ray images, then they can let AI to carry out their own fast diagnosis of breast cancer disease.
Literally the worst fears that we have as a community is that people will recklessly apply ML to things like cancer screening on open websites and cause countless deaths, bankruptcies, needless procedures, etc. How many people went to this website, uploaded images, were told were ok and didn't follow up? How many were told they have cancer and insisted on procedures they didn't need?
The website is totally unaccountable. Totally unregulated. Totally without any of the most basic ethical standards in medicine. Without even the most basic human rights for patients. This is frankly disgusting.
In the US this would have been shut down by the FDA immediately.
We should not be celebrating this unethical "science" that doesn't meet even the most basic of scientific standards or ethical standards.
Unfortunately, it's just a matter of time before someone gets severe complications for whatever outcome your program gets wrong. You have a false negative? Big problem, you just gave someone worse treatment options (if lucky) or led to someone being diagnosed too late and potentially getting metastasized breast cancer. Those bone metastases are quite painful, you know.
False positives are even worse, because they are far far far more likely to happen in practice. Imagine your program telling someone it has a malignant mass (VERY bad wording, only the pathologist can say something is malignant). I speak from experience that this WILL lead to the patient going to the doctor, and the doctor, seeing these very strong words, might want to take a biopsy to confirm the malignancy. These are painful procedures that can and will go wrong, eventually (complication rate is ~1%, which is not a whole lot on its own, but is unacceptable on a known healthy population). If lucky, the biopsy can be done stereotactical, but if unlucky it'll have to be done MRI-guided. You just cost society a few thousand dollars/euros/yuan. And that's if everything goes right, worst case it'll be hospital admission due to complications, like an infection.
Your blog post says you are trying to fight cancer, which is a noble cause. If the tool is not for diagnostical purposes, it's not doing a whole lot in fighting cancer as it is just a play thing then. At the moment, it's more like hindering cancer by taking resources from people who need them and giving them to people who don't need them.
Source: researcher in AI for breast cancer screening/diagnosis.
Of course, this is going to harm some people. Is it possible the number of people it helps is significantly larger than the number of people it harms? If it were so, would you not consider that a reasonable tradeoff?
A slight digression: I find in certain countries (cough cough US), everyone is free to give advice/opinion on anything, except for medical/legal matters, which are considered sacred and so no one is free to give any advice/opinion on those. I saw a person scolded by HR for literally saying "make sure you stay well hydrated". This is madness! Medical professionals should not have a monopoly on advice concerning hydration.
The proper response to being scolded by HR for saying that is to ask what the person's username is, so you can speak to their manager.
Then, when you speak to the manager, ask them not to have their report telling people not to do things that HR doesn't have authority over (coworkers discussing health topics like that are not within the scope of HR).
Is there any possible way that a test that is not entirely reliable can be used for screening? Like, couldn't they have a radiologist look at the photo to confirm the result?
Yeah you could do that and it's happening in practice. Many countries implement double reading where two radiologists look at it. In the USA, single reading + computer-aided diagnosis is quite common using commercial, FDA-approved software.
> just a matter of time before someone gets severe complications for whatever outcome your program
If you are going to indirectly call someone out for publishing code which you deem severely detrimental to human health, you better back your stuff up.
> You have a false negative? Big problem
How does one get a false negative from multi-class probabilities? There aren't any...
> you just gave someone worse treatment options (if lucky) or led to someone being diagnosed too late and potentially getting metastasized breast cancer. Those bone metastases are quite painful, you know.
An image is already checked by a doctor, before given to the patient. A "False negative" then, is simply the doctor saying they are OK, and the ML confirming this, while both missing cancer. If doctor said, not OK, then no patient will take an OK from a website and be diagnosed too late. The last sentence is needlessly spiteful. I hope I misunderstood the intent of it.
> False positives are even worse, because they are far far far more likely to happen in practice. [...] I speak from experience that this WILL lead to the patient going to the doctor, and the doctor, seeing these very strong words, might want to take a biopsy to confirm the malignancy.
If doctor says OK, but my automated second opinion says not OK, then I go back to the doctor, and the doctor will either sooth my worries, explaining why the patch is benign. This is the desired outcome for my health. Or the doctor will second-guess their diagnosis, and perform, with my permission, more thorough tests. If tests come back OK, this will sooth my worries and the doctor's worries, and all will be fine. If tests come back not OK, then the application helped save my life.
There is zero reason to believe this app will bamboozle doctors into spending 1000s of dollars over nothing.
> If the tool is not for diagnostical purposes, it's not doing a whole lot in fighting cancer as it is just a play thing then.
This does not follow. There is a significant contribution with this app for fighting and raising awareness of cancer. Low-expertise hospitals use it as a diagnostics aid. People reported getting a second opinion from this app, returning to the hospital, and finding cancer early.
> At the moment, it's more like hindering cancer by taking resources from people who need them and giving them to people who don't need them.
Yes, it really seems you are deeming this app both severely dangerous and hindering cancer research. Without any solid (ML) reasoning and plenty of non-sequiturs. I'd suggest that even if you continue to hold such views, that you apologize to a fellow researcher.
Source: citizen scientist who open-sourced data analysis tools with disclaimers, now widely used for cancer screening, precision medicine, and improving chemo-therapy.
The words “found malicious mass”, disclaimer or not, could be considered a diagnosis. You can probably say “anomaly”, or maybe even “classified as malicious mass”. With you having connections in the US, it is probably worth talking to an expert if you want to keep this online.
Thank you for all your efforts and significant work. The intent and selfless dedication is moving. As grandma would say, talk does not cook rice. As a former student RN caring for patients at UCSF Neurosurgery & Neurology I witnessed a lot of research innovation on the "hardware" side when the "god" of last hope did grand rounds [1] One could only imagine something like this in the early 90s, software as another adjunct layer to the overall care approach. How can clinicians diagnose and treat if the patient has not been admitted into the point of service? Data driven approaches like yours would have saved a lot of $$$/RIP to increase awareness leading to early detection and care access. Pioneers shine a light for others to see, may detractors never have to face an unintended LP. [2]
You know, if people who teach ML classes at universities point out to you that what you're doing is going to be taught in classes as an example of "what not to do and how not to do it", you should probably immediately stop what you are doing and get opinions from experts in the field.
I will repeat what others are say: this is irresponible due to naivete and could be harmful to people. Please consult with experts on how to proceed.
and this is why healthcare is the #1 source of bankruptcy in the US.
Some people believe that every single person must have Mercedes-Benz type of care in the US.
They cannot fathom that some of the plebs (do they even exist for them?) may want to make their own independent healthcare choices, and are willing to accept the risk ... (or can only afford!) a Suzuki.
What is unethical about this citizen science project? What is ethical about keeping it only for yourself, and not sharing it with the world?
You are saying you have the expertise to build a similar product, but releasing it would mean the worst fear of your community?
> people will recklessly apply ML
What are the indications that this is a reckless application of ML?
> How many people went to this website, uploaded images, were told were ok and didn't follow up?
Common sense dictates exactly zero. Their follow up was taking their images and getting an automated second opinion. Either a doctor already deemed them OK, or a doctor deemed them not OK, in which case, they would not rely on a second opinion, to think they are suddenly OK.
> How many were told they have cancer and insisted on procedures they didn't need?
Again, exactly zero. The app returns probabilities not binary diagnostics. No hospital would do anti-cancer procedures on a patient without cancer, even when they insist, because some website, friend, or religious leader told them so.
> The website is totally unaccountable.
Good. Or make the good-faith open-source project accountable and liable? That would simply mean shutting it down. No more diagnostics help for low-expertise hospitals: not good at all.
> Totally without any of the most basic ethical standards in medicine.
List a basic ethical standard in medicine which this project runs afoul of.
> basic human rights for patients
What right is that? The right not to upload your images to a site of your choosing? I thought human rights include self-determination, and keeping possession of your imaging to do however you see fit.
> In the US this would have been shut down by the FDA immediately.
But is that a good, ethical thing? Or simply that red tape and authority in US does not allow for such projects?
> We should not be celebrating this unethical "science" that doesn't meet even the most basic of scientific standards or ethical standards.
You should not talk about ethics or science, when you did not do even a proper evaluation of the work of a fellow scientist.
> I can't believe this is getting upvoted here.
Awaiting your work on cancer research and ML. Post it here. If devoid of ethical issues, and strongly scientific, it will also be upvoted and celebrated. Or is your major contribution going to be a snipe at someone who actually contributed?
I share your sentiment, people are focusing on successes too much but not scrutinise what potential outcomes false negatives in software like this can have.
> In 2018, a programmer named “coolwulf” started a thread about a website he had made. Users just need to upload their X-ray images, then they can let AI to carry out their own fast diagnosis of breast cancer disease.
Literally the worst fears that we have as a community is that people will recklessly apply ML to things like cancer screening on open websites and cause countless deaths, bankruptcies, needless procedures, etc. How many people went to this website, uploaded images, were told were ok and didn't follow up? How many were told they have cancer and insisted on procedures they didn't need?
The website is totally unaccountable. Totally unregulated. Totally without any of the most basic ethical standards in medicine. Without even the most basic human rights for patients. This is frankly disgusting.
In the US this would have been shut down by the FDA immediately.
We should not be celebrating this unethical "science" that doesn't meet even the most basic of scientific standards or ethical standards.
I can't believe this is getting upvoted here.