Yup. This is exactly the same problem as the self-driving car. The tech is not 100% reliable. There are going to be incidents. When an incident happens, who takes the blame and what recourse is available? Does the corp using the AI simply eat the cost?

See also https://www.londonreviewbookshop.co.uk/stock/the-unaccountab...