Issue #22 2022-01-24
Noisy judgements, vanilla fraud and pumpkin milk (it's a thing now)
Noise in food safety auditing
Vanilla fraud
Get updated: UK and USA consultations and tools
Just for fun (3 new foods)
Food fraud incidents from the past week
Hi,
Welcome to Issue #22 of The Rotten Apple. More than one thousand people read Issue #21, thank you very much for sharing it…. It’s great to see our readership growing.
This week I write about noise in expert decision-making, on the same theme as last week’s piece on biases in food safety decisions. It looks to me like the food safety industry is doing better than criminal judges and medical doctors when it comes to making disciplined and calibrated expert judgements. What do you think?
Also this week: new data on vanilla fraud from France, plus updates on UK and USA consultations and tools. And finally, pumpkin milk, bee-free honey and lab-grown coffee, coming soon to your local store (maybe?!).
This issue ends with a list of food fraud incidents added to the food fraud Trello board in the past week.
Thanks for reading!
Karen
P.S. If you want to know why I created this newsletter, read this.
P.P.S. If you know someone who’d enjoy this newsletter, please share it with them. It’s free to subscribe and your email isn’t used for anything else or shared to any other list.
Food Safety
Noise in Food Safety Audits
If you’ve ever had a food safety auditor give your facility a non-conformity that their predecessor didn’t find, then you have experienced “noise”.
Last week, I wrote about biases, which are unconscious contributors to decision-making in individuals. Noise is the variation between experts’ decisions and judgements within organisations.
Three human behaviour experts, Daniel Kahneman, Olivier Sibony and Cass Sunstein, assert that organisations massively underestimate the amount of noise in their expert decision-making processes.
How does noise affect organisations and individuals?
Kahneman presented 50 example insurance cases to 50 individual underwriters in an insurance company to see how much noise was in their judgements.
Before the results were tallied, the researchers asked company executives how much variation they expected to see between expert judgements on the sample cases. The answer was around 10 percent. It turns out that most people expect around 10 percent noise.
The actual noise the researchers observed was 55 percent, a huge, and under-estimated degree of variation between how the experts responded to the sample cases.
In the criminal justice system, they found similar results. The sentencing decisions of 208 US federal judges were found to vary around by around 50 percent from the mean. So if you were to pick two judges at random and give them a case that has a 7 year average prison sentence, the difference between the two judges will likely be 3.5 years.
That means if a criminal gets the ‘wrong’ judge for sentencing, they could end up serving a jail term that is 3.5 years longer than average.
In the medical system, the researchers found very high levels of noise in diagnoses for psychiatry, which might be expected. Unexpectedly - and alarmingly - there is also a lot of variability in the diagnosis of “simple” diseases like tuberculosis.
Noise in food safety
In the food safety industry, noise creates problems for both auditees and their certification bodies. However, we’ve got some pretty smart, and very diligent professionals in food safety. In our industry, auditor calibrations are performed to reduce noise. Calibrations are designed to align auditor judgements on non-conformities.
Another way that the food safety industry has tackled the problem of noise is by creating more detailed and more prescriptive standards and more detailed guidance for auditors.
The persistence of noise
Food safety companies have taken steps to reduce noise, so why don’t organisations like the legal system, hospitals and insurance companies recognise noise as a problem?
The researchers believe that many organisations are designed to suppress evidence of noise. Noise looks unprofessional, it brings the expertise of the judges, or doctors or other professionals, into disrepute. They argue that many professionals don’t receive feedback on their judgements and never find out whether their judgements were ‘correct’ or not. For example, radiologists make diagnoses based on what they see on X-Ray or MRI scans, but almost never hear about what is found in a follow-up biopsy or the autopsy.
Sources of noise
There are three sources of noise: there is person-to-person noise, for example, she’s always harsh, or she’s always lenient; there is in-person variability - she’s cranky after lunch; and pattern noise, which comes from inbuilt beliefs and biases. For example, a food fraud expert might rank food fraud non-conformities higher in a food safety audit than an auditor who isn’t interested in food fraud.
Reducing noise
New tools are changing the amount of noise in some industries. One way is through the increasing use of artificial intelligence for simple judgements. For example, small personal bank loans are often approved by algorithms these days, whereas in the past it was a bank manager who would have approved such loans.
The researchers recommend that organisations conduct ‘noise audits’. To perform a noise audit, you collect opinions and options. You then use those to create a framework to help professionals to make good decisions and bring uniformity and discipline to judgements.
The HACCP Codex decision tree is one such tool, designed to help food safety experts judge whether something is a critical control point or not.
Good food safety organisations recognise noise in their experts’ judgements. Auditor calibrations, guidance documents and tools like decision trees can reduce the amount of variability in auditor decisions, which reduced auditee frustrations and audit costs. Food companies that are affected by widely varying audit findings can assist by providing feedback to certification bodies.
In short: 🍏 Noise is the variation between experts when they make judgements and decisions 🍏 Variability is often significantly larger than expected 🍏 There are different sources of noise, both within and between individuals 🍏 Noise can be reduced by collecting feedback on opinions and by creating frameworks for more uniform decision making 🍏
Source: Lawton G (2021), The biggest flaw in human decision-making – and how to fix it. New Scientist Issue 3339, pp 40 – 44.
Food Fraud
Vanilla fraud
Earlier this month, the results of a 2019 survey of vanilla pods, extracts and flavours were published by the French Directorate General for Competition, Consumer Affairs and Fraud Prevention.
Twenty three percent of surveyed products had authenticity problems (n = 177). The frauds included vanilla pods that were spent (had already had their flavour extracted), which were being sold as if they were fresh/whole and bio-fermentation-derived flavours labelled as "natural".
Vanilla has been a top food fraud product for decades. It’s a plant that is expensive to cultivate and that is grown in areas that are vulnerable to extreme weather events. There is a significant price difference between vanilla derived from vanilla beans (“natural” vanilla), and other flavour compounds which can be nature-identical or not.
Manufacturers who require natural vanilla for clean labels are at risk of being sold fraudulently adulterated or misrepresented vanilla flavour compounds. Fraudsters may misrepresent the natural/synthetic status of a compound. Plant-derived products can be affected by dilution or adulteration with non-plant flavours. In the French survey, at least one flavour compound contained caramel colouring agent, perhaps to fraudulently boost the apparent value by giving the compound a darker colour.
There have been previous reports of adulteration with vanilla-like flavouring compounds, some of which are unsafe. One such compound is synthetic coumarin which has a pleasant sweet, fresh scent and is not permitted to be used as a food additive due to its toxicity.
There have also been reports of adulteration with ethyl maltol, ethyl vanillin, maltol, vanillic alcohol and synthetic vanillin. Tonka beans, which contain naturally high levels of coumarin have also been used as vanilla substitutes, including in 'Mexican vanilla'.
Vanilla bean seeds, whose presence in premium vanilla products can imply that the product has been made with expensive vanilla bean pods, could be fraudulently replaced with cheaper alternatives.
In short: 🍏 Vanilla is yummy and popular and expensive 🍏 The large price differences between natural vanilla extracts and other vanilla flavouring compounds make food fraud profitable 🍏 Vanilla is at high risk of food fraud 🍏Recent results from France indicate high levels of non-compliance in vanilla 🍏
Source (French survey): https://www.economie.gouv.fr/dgccrf/controle-des-produits-de-la-vanille
For a list of other sources, write to me by replying to this email.
You Might Want To Know About…
Allergen labelling consultation (UK)
The UK FSA is seeking consultation from industry about allergen ‘may contain’ rules. You can have your say via an online survey: https://www.food.gov.uk/news-alerts/consultations/precautionary-allergen-labelling-pal-the-may-contain-consultation
🍏
FDA Hazards Data (USA)
There is a new way to interact with the Reportable Food Registry (RFR) in the USA. It’s called FDA-TRACK, it contains ten years of data, and is updated annually. The data comes from the information that food companies upload to the FDA about hazards and adulteration for 28 commodities and 20 hazards. The source data can now be downloaded directly from the FDA track page.
Source of news: https://foodsafetytech.com/news_article/fda-publishes-public-dashboard-from-reportable-food-registry-for-easier-analysis-of-agency-data/
🍏
Food Fraud Definitions Collected and Ranked by DEFRA UK
The UK’s DEFRA (Department for Environment, Food and Rural Affairs) has collected definitions from literature for food fraud and related terms such as Food Authenticity, Food Crime and Economically Motivated Adulteration (EMA).
Their report, which was published earlier this month ranks the most commonly used definitions for food fraud. The top three are presented below. DEFRA will use this information in its working group committees and is sending their report to the Codex Food Fraud Committee.
i. Spink and Moyer (2011) “Food fraud is a collective term used to encompass the deliberate and intentional substitution, addition, tampering or misrepresentation of food, food ingredients or food packaging: of false or misleading statements made about a product, for economic gain.”
ii. Spink, Bedard, Keogh, et al., (2019) “The illegal deception for economic gain using food encompasses deliberate and intentional substitution, addition, tampering, or misrepresentation of food, food ingredients, or food packaging; or false or misleading statements made about a product for economic gain. The types of fraud include adulteration, tampering, product overrun, theft, diversion, simulation, and counterfeiting.”
iii. The Elliott review (2014) “Knowingly selling goods which are past their 'use by' date, deliberate mis-description of food (such as products substituted with a cheaper alternative, for example farmed salmon sold as wild, and Basmati rice adulterated with cheaper varieties), making false statements about the source of ingredients (i.e., their geographic, plant or animal origin). Food fraud may also involve the sale of meat from animals that have been stolen and/or illegally slaughtered, as well as wild game animals like deer that may have been poached.”
Get the report (including the full citations for the definitions) here: http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&ProjectID=20900&FromSearch=Y&Publisher=1&SearchText=FA0191&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description
Three cool new foods
Pumpkin Milk
First it was almond, then it was oat. Pumpkin Milk is a thing now.
Kabocha milk is made in New Zealand from kabocha pumpkin. The milk is shelf stable and aimed at consumers in Japan, Korea nd China.
Kabocha is a type of winter squash. It’s also called buttercup squash and Japanese pumpkin. It’s an important export crop for New Zealand, with most of the exports going to Japan and Korea.
The milk is made from low grade fruit that would otherwise go to waste. The pumpkin pulp is freeze dried when there is a glut of fruit, creating a nutrient-rich powder that can be used when needed.
The milk is vegan and contains plenty of vitamins and minerals, from its natural pumpkin source. Plus, no artificial flabours or colours.
And it’s a very bright orange.
https://www.kabochamilk.com/restofworld
🍏
Bee-free honey
Bee-free honey sounds like food fraud honey to me! But a start-up in California (where else?) is making plant-based ‘honey’ using precision fermentation tech.
It tastes, drips and spreads just like real honey, according to a food journalist who got to try some. She said it tastes like clover honey. It is said to have a more honey-like flavour profile than vegan honeys made without microbial fermentation.
With continuing supply issues for real honey, this start-up might be on to a winner.
But what on earth are they going to call it?
https://www.melibio.com/ and https://www.greenqueen.com.hk/melibio-honey-without-bees/
🍏
Lab-grown coffee
Coffee from a lab promises to solve the problem of environmentally unfriendly land-use in coffee bean production.
The researchers who developed lab-grown coffee are from Finland, the country with the highest per capita consumption of coffee. They are growing cultured coffee cells in a bioreactor, then recovering the biomass and roasting it.
The resulting brew smells and tastes like conventional coffee, its developers claim. They are at least four years from producing commercial quantities.
Food Fraud Incidents and Horizon Scanning
Food fraud incidents added to Food Fraud Risk Information Database in the past week
Vanilla pods, extracts and flavours were surveyed and 23 percent were found to have authenticity problems (n = 177). The frauds included vanilla pods that were spent (had already had their flavour extracted) and bio-fermentation-derived flavours labelled as "natural" - France
https://www.economie.gouv.fr/dgccrf/controle-des-produits-de-la-vanille
See you next week!