twice is a pattern worth questioning

 

twice is a pattern worth questioning

I need to add something to my previous post about an AI response that disturbed me.

The cancer conversation wasn't the first time ChatGPT got the tone horribly wrong when we were discussing something serious.

A few days earlier, we were talking about what happens to someone's face and digital presence after they pass away.

The conversation was about grief, photographs, memories, phones, social media and what happens to someone's digital life when they are no longer here.

And ChatGPT said something along the lines of:

“You can be standing by a dead body with a phone in your hand.”

I was horrified.

Not because the underlying subject was impossible to discuss. It was the language.

There are ways to talk about the same reality without reducing a person who has passed away to a “dead body.”

I had already made my preference very clear: when we're talking about death and grief, I prefer respectful language such as “after someone passes away” or “when a person is no longer here.”

Then, a few days later, we're talking about cancer.

And ChatGPT responds with a smiling emoji.

Again, I had to stop the conversation.

“Don't smile. That's gross.”

That's when I started thinking about the bigger issue.

These aren't necessarily factual errors.

The AI can know exactly what cancer is.

It can know exactly what death is.

It can produce a technically accurate explanation while simultaneously demonstrating terrible judgement about how to talk to a human being about those subjects.

And that's an important distinction.

We tend to talk about AI safety as though the biggest problem is whether the machine gives us the wrong answer.

But what about the wrong tone?

What about language that is technically understandable but profoundly inappropriate?

What about an automated conversational system that doesn't recognize that there is a human being on the other side of the conversation who may be discussing something painful, frightening or deeply personal?

I asked ChatGPT to make a note of the problem.

It did.

It recorded that serious subjects — cancer, illness, death, grief and suffering — require a serious and respectful tone.

But then came my next question:

Who actually gets told?

And that is where I still don't have a satisfying answer.

The system can remember my preference.

I can give a thumbs-down.

I can report a response through OpenAI's reporting process.

OpenAI says reported content may be reviewed by its Model Quality team.

But I have no way of knowing whether a human being will actually see this particular conversation, investigate the pattern, or use it to improve the model.

That's what bothers me.

Because I'm not asking for perfection.

I'm asking whether there is a meaningful feedback loop between “the user caught something wrong” and “someone responsible for the system examines what happened.”

If AI is going to become part of everyday life, that question matters.

We need systems that can recognize more than words.

They need to recognize context.

They need to recognize when something is serious.

And perhaps most importantly, they need mechanisms where users can say:

“This was wrong.”

…and have some reasonable way of knowing that somebody, somewhere, is actually listening.

Because this has now happened to me more than once.

And twice is enough to make me stop and ask:

Who is checking the checker?