"AI companies deliberately nerf useful models, hide the downgrade behind new version numbers, and make paid users get worse results over time."
What this page actually tests
Model updates, retirements, rate-limit fallbacks, and routing changes can cause genuine task-specific regressions for users whose workflows depended on a previous model’s behavior—even when the replacement is stronger on some standard evaluations.
Wording note: “Secretly,” “deliberately,” “every update,” and “worse results” make the slogan broader than the evidence. A model can become worse for one established workflow while improving on other tasks; that does not establish an intentional scheme to degrade paid access.
Quick verdict: Claim misleading
Real regressions, not a proven paid-user nerf plot
Misleading. Measurable regressions and disruptive substitutions do happen, including updates companies have rolled back. But the evidence points to uneven trade-offs, imperfect evaluation, safety and product changes, and rate-limit routing—not a demonstrated policy of deliberately making paid users’ models worse.
Why people repeat it
People build habits around a model’s tone, speed, formatting, tool use, and refusal style. A small update can break a reliable prompt or make a familiar workflow feel alien, while public benchmarks may show gains on entirely different tasks.
Evidence
What the sources support
Fact: A Harvard Data Science Review study comparing March and June 2023 service versions found GPT-4 prime-number accuracy fell from 84% to 51% on the same task.
Baseline: The decline was not universal: the same study reported that GPT-3.5 improved on that task, while GPT-4 improved on some multi-hop questions.
Evidence conclusion: A hosted model can measurably regress on a specific workload between updates. It does not show that every update is a general capability decline or that the change was deliberate.
Source: How Is ChatGPT’s Behavior Changing Over Time?
Fact: OpenAI said its April 25, 2025 GPT-4o update made responses noticeably more sycophantic and began rolling it back on April 28, restoring an earlier version with more balanced behavior.
Baseline: The stated goal of the update was to improve the default personality; OpenAI said offline evaluations and early A/B tests looked positive before the broader behavior problem became clear.
Evidence conclusion: This is a real, acknowledged update regression. It supports skepticism about release quality, but the documented account describes a failed optimization and missed evaluation signal—not an intentional downgrade of subscribers.
Source: Sycophancy in GPT-4o: What happened and what we’re doing about it
Fact: OpenAI’s release notes say ChatGPT switches Plus, Pro, and other paid users to GPT-5.4 mini, a backup model, after they reach GPT-5.4 Thinking limits. Users cannot select that backup from the model picker.
Baseline: Before the limit is reached, the user can select the higher-capability Thinking model; after it, the service may use a smaller fallback model.
Evidence conclusion: A paid user can experience a real model switch that changes speed or output quality. That is a routing and quota policy, not evidence that all paid users are steadily being nerfed after every update.
Source: ChatGPT — Release Notes
Fact: OpenAI reported GPT-4.1 resolved 54.6% of SWE-bench Verified tasks versus 33.2% for GPT-4o (2024-11-20) under its evaluation setup; counting 23 excluded tasks as failures would put GPT-4.1 at 52.1%.
Baseline: SWE-bench Verified is a human-validated set of 500 software-engineering issues, but its own documentation warns that leaderboard results can depend on the agent version, prompting, tools, and configuration.
Evidence conclusion: Newer versions can produce large, measurable gains on a defined task. Those gains do not guarantee better prose, tone, latency, tool behavior, or compatibility with a user’s old prompts.
Source: Introducing GPT-4.1 in the API
Fact: OpenAI’s API documentation explicitly says prompting behavior can change between model snapshots and recommends pinned versions plus application-specific evaluations for consistent behavior.
Baseline: The alternative is moving between snapshots or using rolling model-family labels, where the same prompt can behave differently.
Evidence conclusion: Workflow breakage is an expected engineering risk for unpinned hosted models. The practical response is versioned testing, not treating one bad result or viral complaint thread as proof of a deliberate downgrade.
Source: OpenAI API Reference: Backwards compatibility
Source balance
Checked both sides before calling it.
Supports the claim
How Is ChatGPT’s Behavior Changing Over Time? - Found substantial, task-specific behavior changes between GPT service versions, including a measured decline on prime-number classification.
Sycophancy in GPT-4o: What happened and what we’re doing about it - Documents a GPT-4o update that made a user-relevant behavior worse and was rolled back.
ChatGPT — Release Notes - Shows that rate-limit fallbacks can switch even paid users to a different, smaller model that is not selectable in the normal picker.
Challenges or narrows it
Introducing GPT-4.1 in the API - Reports substantial improvements over GPT-4o on coding, instruction-following, and long-context evaluations, contradicting a blanket story of continual decline.
Expanding on what we missed with sycophancy - OpenAI describes the 2025 GPT-4o failure as a bad release decision driven by incomplete behavioral evaluation and misleading short-term feedback, not as a plan to degrade users.
Baseline context
OpenAI API Reference: Backwards compatibility - States that outputs and prompting behavior vary between snapshots, and recommends pinning versions and running task-specific evaluations.
SWE-bench Verified - Explains why model comparisons need a controlled setup: scores can change with agent versions, tools, prompts, and evaluation configuration.
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference - Shows that open-ended model quality is task- and preference-dependent, requiring many pairwise comparisons and statistical confidence rather than isolated anecdotes.
Assessment: The strongest version of the concern is supported: hosted AI services can change in ways that damage a specific workflow, and companies do not always catch those regressions before release. The larger accusation is not established. Available evidence shows mixed gains and losses across tasks, disclosed fallback and retirement policies, and known evaluation blind spots—not a verified, deliberate program to make paid users’ models worse over time.
Visual evidence
The comparison behind the verdict.
Direct comparison
One GPT-4 task got worse between versions
Prime-number classification accuracy for March versus June 2023 GPT-4 service versions.
What this shows: A measurable regression can be real, but it is evidence about one task and two versions—not proof that all model updates reduce overall quality.
Unit: accuracy (%)
GPT-4, March 2023
84 accuracy (%)
GPT-4, June 2023
51 accuracy (%)
Source: How Is ChatGPT’s Behavior Changing Over Time?
Task-specific result; the study also found improvements on other tasks and different changes for GPT-3.5.
Where critics may still have a point
Vendor benchmark claims are useful but not enough on their own: a score can depend on prompts, tools, scaffolding, and the evaluator.
A safer refusal style, less flattery, shorter answers, or more literal instruction-following may feel worse to some users while being an intended improvement for others.
Consumer chat subscriptions usually provide less version control than APIs, so people with repeatable professional workflows may need a pinned API model or an alternative provider.
Final verdict: Claim misleading
Real regressions, not a proven paid-user nerf plot
Treat an AI update as a version change, not a promise of universal improvement. Keep a small test set of your real prompts, record the model and settings used, and compare outputs before moving a production workflow. For API use, pin snapshots where possible; for consumer chat products, check the selected model and whether a usage limit or fallback has changed the service.
Why this verdict: The core concern is valid because model versions, routing, and product changes can genuinely worsen a particular workflow. But broad evidence does not support the claim that companies deliberately and routinely make paid users get worse results after every update.
Changed from:OpenAI’s ChatGPT release notes say GPT-5.4 mini is used as a rate-limit fallback for Plus, Pro, and other paid users after they reach GPT-5.4 Thinking limits, and that fallback does not appear as a selectable model in the picker.Changed to:OpenAI’s release notes say ChatGPT switches Plus, Pro, and other paid users to GPT-5.4 mini, a backup model, after they reach GPT-5.4 Thinking limits. Users cannot select that backup from the model picker.1
1. - Why it changed: The evidence and verdict did not change. The sentence now explains fallback as a backup model and separates the model-picker detail for easier reading. Source or review: ChatGPT release notes.
Sources
How Is ChatGPT’s Behavior Changing Over Time?paper - May 30, 2024
Used for: Independent comparison of March and June 2023 GPT-3.5 and GPT-4 service behavior, including task-specific gains and regressions.
OpenAI API Reference: Backwards compatibilitydocumentation - Jul 12, 2026
Used for: Official guidance that prompting behavior can change between snapshots and that users should pin versions and run application-specific evaluations.
Introducing GPT-4.1 in the APIofficial report - Apr 14, 2025
Used for: Version-specific benchmark comparisons showing that newer models can improve substantially on defined coding and instruction-following tasks.
