Fresh juice

2024-04-04

ChatGPT has really become more "dumb" lately

Recent research from Stanford and UC Berkeley found declines in performance for OpenAI's ChatGPT conversational AI. The results validate user complaints of reduced effectiveness in recent weeks.

ChatGPT's popularity has surged since launching in November 2022. But users began reporting more spelling errors, lapses in context, and unreliable responses this June. Site traffic and time spent also dipped for the first time.

Controlled experiments by the researchers confirmed major drops in ChatGPT's abilities between March and June. Performance identifying prime numbers fell from 97.6% to 2.4% accuracy. Solving a basic math problem declined from 52% to 10% accuracy.

The predecessor GPT-3.5 model meanwhile showed gains in the same areas. This contrasts OpenAI's claims that each ChatGPT version outperforms earlier ones. There are widespread complaints on OpenAI's developer forums about wavering reliability.

While reasons are unclear, the findings undermine faith in purported ongoing improvements. OpenAI VP of Products Peter Velinder denied intentionally degrading capabilities, tweeting "No, we did not make GPT-4 dumber."

He suggested heightened use exposed more flaws and that GPT-4 remains in development. But OpenAI faces growing pressure for an official response and demonstrable upgrades amidst competition from Google's Bard and upcoming alternatives.

Restoring trust will require transparency on ChatGPT's inconsistencies and concrete steps to enhance stability. With AI chatbots going mainstream, precision and robustness are critical.

OpenAI must confront the performance decline instead of dismissing it as increased scrutiny. Sustaining lead in conversational AI entails acknowledging and addressing the problems users encounter firsthand.

Share with friends:

Write and read comments can only authorized users