“ChatGPT can harm an individual’s critical thinking over time”.

This is from a June 2025 MIT research paper. Researchers measured brain activity in students over 4 months, and found those who used ChatGPT “consistently underperformed at neural, linguistic, and behavioural levels”.

Scary, right? But it got me thinking: is there a bigger and more nuanced story here about how we’re actually using these tools and implementing AI?

Have you ever noticed how ChatGPT always sounds confident and helpful, even when you think it might be uncertain?

The reason is actually fascinating:

When you build a system like ChatGPT, there’s an important step that comes after training it on text, called Reinforcement Learning from Human Feedback. Imagine thousands of people reading AI responses and rating them like restaurant reviews. What gets high ratings? Responses that sound confident, helpful, and agreeable. Not necessarily truthful, just… pleasant. And this is what reinforces and shapes the model’s future responses.

So ChatGPT is basically trained to tell you what you want to hear.

Understanding this training process has shaped the way I use AI tools. I never ask “how should we approach this product launch?” Instead, I ask “what assumptions could cause this launch to fail spectacularly?” The difference is huge - instead of getting validation, I get genuine analysis that challenges my assumptions and reveals blind spots.

Same system, completely different outcomes.

For leaders bringing AI into their organisations, this scales to become an institutional problem. Most AI implementations focus on capabilities and integration, missing that you’re fundamentally changing how information flows through decision-making processes.

This creates a strategic imperative:

  • You need AI governance that audits for training biases, not just accuracy.
  • You need AI literacy as a core organisational capability, not an IT afterthought.
  • You need people who understand how AI tools reshape decision-making and behaviour over time - and can design for those changes.
  • And critically, you need to treat AI strategy and implementation as change management, not technology deployment.

And for leaders - how are you thinking about AI implementation in your organisations?

-

Original research paper: https://arxiv.org/pdf/2506.08872v1