132 | Is ChatGPT Getting Dumber? | Fools Foods | Sloths and Snails |
Chamomile tea for dogs?!
Is ChatGPT getting dumber?;
Snails and sloths - food labelling changes by the USDA;
Fools Foods for 1st April;
Two long reads for your holiday pleasure
🎧 Listen (now for everyone!) 🎧
Hi!
Welcome to Issue 132 of The Rotten Apple. Today is a holiday in many countries, including mine, so I am departing from our usual format this week.
This week’s issue is a no-work zone.
Instead of the usual four sections, I’ve posted an off-topic article about ChatGPT. This is a follow-up to last year’s super-popular post about AI writing in food safety. In that post I reported on the disturbing results I got when I asked ChatGPT to help me write an article on an obscure type of food poisoning.
This year I ask: Is ChatGPT getting dumber?
I’ve also got four fun April Fools foods for you, plus a quote about the US food regulatory system that made me laugh out loud.
There’s no food safety or food fraud news this week, but don’t despair. We’ll catch up on the food fraud news next week (for paying subscribers), so you don’t miss a thing. Because being a paid subscriber means getting access to high-quality food fraud updates…. and supporting this robot-free publication.
Thank you!
Karen
P.S. Need more info about paid subscriptions? Learn more here. Or….
Is ChatGPT getting dumber?
Since ChatGPT sprung into our collective consciousness in late 2022, it’s been blowing our minds with its seemingly endless use-cases. Want to plan a kid’s party? Ask ChatGPT! Create a PowerPoint deck for work? ChatGPT! Write an obituary for your dead mother? Oh boy, yes.
But lately people are asking: Is ChatGPT getting dumber?
Since the middle of last year, there have been persistent complaints on social media that ChatGPT is more prone to making mistakes, generating less effective computer code for programmers and less likely to come up with original-sounding ideas than when it first launched.
“The responses feel a little cagier at times than they used to,” said one commentator. “I assume it’s trying to limit hallucinations in order to increase public trust in the technology.”
There are certainly more disclaimers and caveats in the answers it gives these days. Perhaps these templated caveats undermine the perceived value of the software’s “thought processes”? pondered another.
Other users have complained that, compared to the early days of the app, current answers lack nuance and complexity. Some complain that the language has been “dumbed down” too far.
When it comes to creating computer code, all the software guys seem to agree. “There’s no doubt that it’s gotten a lot worse on coding,” cried internet users on X.
Some people are blaming the censorship systems added to the model soon after it was launched, after the app gave problematic answers to sensitive questions in its earliest iterations. “In essence, it's been lobotomized to prevent it from generating porn or saying the N-word.” said another X user.
And it’s not just the free version, paid subscribers are cancelling their subscriptions to ChatGPT, citing the increasing “dumbness” of the paid version.
So what’s going on here? Is ChatGPT performing worse than it did before? And is this an unintentional side-effect of the model’s generative processes or the result of intentional ‘tweaks’ from its owners?
Neither, said ChatGPTs developers. When the complaints first began to surface, OpenAI (owner of ChatGPT)’s vice president of product jumped in to defend the app, saying it was in a process of constant improvement and that the complaints were an unfortunate side-effect of its runaway success.
ChatGPT was so dazzling when we first saw it, he said, that it was destined to become disappointing over time. The more you use a new app, the more likely you are to notice issues you didn’t see before. It was, he said, simply a case of familiarity breeding contempt.
What does ChatGPT ‘think’ about all this? I decided to go straight to the source.
Hilariously, when I asked it directly, it got all defensive, explaining it’s having trouble “keeping up”. Awww, bless.
🙋♀️ Me: So everyone says you are getting dumber. What gives?
🤖 ChatGPT: This isn't about some sudden meltdown in technology. It's more about how AI, like ChatGPT, works over time. You see, it relies on data and clever algorithms to chat with us. But as things change online, with new stuff popping up all the time, it can make things tricky for ChatGPT to keep up. There is some objective evidence to support the ‘getting dumber’ hypothesis. Researchers from Stanford and UC Berkeley measured the performance of two versions of the app, GPT-3.5 and GPT-4, over a three month period between March and June 2023. They reported that both versions became worse at certain tasks between the start and end of the study period.
ChatGPT’s ability to identify prime numbers, produce error-free computer code and follow user instructions were measurably worse in June compared to March. The researchers also described “a drop in GPT-4’s amenity to follow chain-of-thought prompting.” In addition, they noticed GPT-4 was less willing to engage with sensitive questions in June compared to March.
The phenomenon of unpredictable changes in performance is known as “drift”. The researchers told the Wall Street Journal that they were not surprised to find evidence of drift in their study, but were very surprised by the speed at which it appeared to be happening.
My theory, that language models which are trained on internet content are now getting trained on content that was written by their predecessors, hasn’t gained any traction online (so far!). Training new iterations of large language models on low-quality content written by earlier versions of large language models must, I imagine, make them become progressively worse.
Some people call this enshittification. (Yes that’s a word, in fact it was the American Dialect Society’s 2023 word of the year!)
Or perhaps the robots have been infected by their interactions with humans.
A person who calls herself a formerly miserable but now very happy academic on X, described how an unnamed AI expert explained that models are getting dumber because of their interactions with people. Are we, perhaps, ‘training’ the models to be somehow less helpful than they were before?
But if we are making ChatGPT dumber, there’s every possibility that it’s doing the same to us. A recent paper published in the International Journal of Educational Technology in Higher Education described the impact of ChatGPT on students' academic performance and creativity. The authors measured the effects of ChatGPT usage on university students' procrastination, memory loss, and academic performance.
Students who used ChatGPT during the study period were more likely to be affected by procrastination, memory loss and decreasing academic grades than their peers, though the researchers conceded that perhaps ChatGPT was more appealing to students who were already struggling.
It seems like - anecdotally at least - ChatGPT is losing its shine for some users. For me, I’ll keep using it to write family obituaries (joking!), and will continue to keep it away from my food safety writing.
Sources:
Chen, L., Zaharia, M., Zou, J. and University, S. (n.d.). How Is ChatGPT’s Behavior Changing over Time? [online] Available at: https://arxiv.org/pdf/2307.09009.pdf.
Abbas, M., Jam, F.A. and Khan, T.I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, [online] 21(1), p.NA–NA. doi: https://doi.org/10.1186/s41239-024-00444-7.
🍏 ChatGPT is free software, available here: https://chat.openai.com/ 🍏From the archives…
No Robots Allowed: Why AI-powered Writing Doesn’t Belong in Food Safety Publications
Regulatory News
I promised this issue would be a ‘no work zone’, but I couldn’t resist sharing a comment from an industry commentator frustrated about the timing of coming labelling changes which are intended to close a loophole in the rules for ‘Product of USA’ claims for meat.
🐌 As fast as a snail stuck in superglue.
🦥 As proactive as a sloth hanging upside down in an opium den.
Jonathan Needham, via LinkedIn, talking about the speed of the USDA’s meat labelling rule changes.
April Fools Foods
It’s the 1st of April and I’m excited to see what cute and crazy April Fools Day joke products will be announced this year. Here are some of my favourites from 2023.
Milliways Chewing Gum announced three new flavours: Plastic Bag, Credit Card and Plastic Straw. Milliways is a plastic-free chewing gum brand and was trying to draw attention to its competitors’ plastic ingredients.
Nestle announced the classic 4-finger KitKat had been relaunched with a thumb.
Scottish malt whiskey company Glenlivet announced it would launch 1.5 L cardboard cask packaging (known as a ‘goon-bag’ in Australia) for its 12 year old single malt whiskey. The product, they said, would be called Goonlivet.
The dog food company Scratch launched a range of relaxing teas for dogs, including one with “the faintest touch of freshly-mown grass” and a chamomile tea with kangaroo kibble flavour.
Two Good Long Reads
Because it’s a holiday today in my country, here are a couple of interesting articles I thought you might enjoy, instead of the usual food fraud news.
The inspiring life story of the man who dreamed up cultivated meat in a World War II concentration camp
“Incensed that medicine was unable to save [his young wife], Willem van Eelen fell into a deep depression. He threw himself deeper into his work trying to use stem cell research to make a modern meat product—fueled by an obsession with immortality he’d come to embrace following the death of his wife.
“He lost the mother of his child,” his daughter says, and “he got into this total belief that dying is something that you shouldn’t do.” He would later tell one business partner that he thought he would live to be at least 120.”
Read: The idea for lab-grown meat was born in a prisoner of war camp (qz.com)
Artichokes and the mafia
The Sicilian American mafia has long been involved in doubtful business operations. They were even involved in the earliest days of the (then) highly profitable American artichoke industry.
“Even then, [in 1935 New York], some observers found LaGuardia’s claims about a vast artichoke underworld bizarre. “It is impossible not to conclude the world today is a bit mad,” a reporter covering the ban for the New York Herald Tribune wrote.”
Read: In 1930s New York, the Mayor Took on the Mafia by Banning Artichokes - Gastro Obscura (atlasobscura.com)
What you missed in last week’s email
Case study: How massive quantities of soybeans and corn turned “organic” in the blink of an eye;
Banana extinction - a new hope;
Are egg shells weaker than before?;
The raw chicken challenge.
🍏 That’s all for this week,
I’m off to make a chamomile tea for my pet sloth. Bye!