Eating disorder helpline fires humans, employs chatbot

It's 2023 and people still aren't getting the message.

You CANNOT replace human beings with any sort of chatbot for anything more complicated that a restaurant booking service and expect it to cope.

You ESPECIALLY shouldn't think that firing the humans who operate your Eating Disorder helpline (for people with complex mental health issues around eating) and replacing them with an “AI Chatbot” is going to be anything other than an actual disaster.

And yet, here we are: https://www.theguardian.com/technology/2023/may/31/eating-disorder-hotline-union-ai-chatbot-harm

If you don't want to read the whole article (and I think you should to get the full idea of how bad it got) here's the tl;dr:

• The humans who operated a help line for people with eating disorders decided to unionise.
• The organisation offering the help line didn't like that, so they fired the humans and replaced them with a chatbot called “Tessa”
• “Tessa” immediately started offering tips to people accessing it on how to lose weight. Yup, on an eating disorder help line.
• The organisation offering the help line ended up pulling “Tessa” from service

So now there is no helpline at all (a decision the organisation in question says was three years in the making).

Can we please just make one thing very clear. So called “AI” isn't. Even the most advanced Large Language Model is a very fancy Auto-complete that can do very clever things, but is about as “intelligent” as a brick. It certainly is no where near capable of being empathetic or handling complex human emotions or motivations.

Tech is awesome, tech is great, tech can do amazing things, one thing it absolutely cannot do is fix human beings mental health.