The quality of "I don't know": designing AI that people trust with their kids' food

AllergySnap gives one of four answers when you scan a product: Safe, Warning, Danger, or Unknown. The first three are what people download the app for. The fourth is what makes them keep using it.

This is counterintuitive if you think of an AI product as an answer machine. An answer machine that says “I don’t know” looks broken. But a parent checking whether a snack is safe for a child with a nut allergy is not looking for an answer machine. They are looking for something they can trust more than their own tired reading of a tiny label in a busy aisle. And trust, in this domain, is built on three things that have nothing to do with recognition accuracy.

Show the evidence, not just the verdict. Both AllergySnap and FoodGrader highlight the specific ingredients that drove the result: the synonym for milk protein buried in the list, the onion powder listed fourth, the “natural flavors” that may hide a FODMAP trigger. A verdict without a reason cannot be checked by the person holding the box. A verdict with the reason can be checked in two seconds, and every time it checks out, trust goes up.

Be explicit about uncertainty. Labels are photographed at angles, in bad light, with glare. Products change formulation. Some information is simply not printed on the packaging; FODMAP content is inferred from ingredients, never stated. A product that always produces a confident answer under those conditions is lying some of the time, and its users will eventually find out in the worst possible way. FoodGrader labels its FODMAP flagging as best-effort. AllergySnap says Unknown and tells you to check manually. Neither is a weakness; both are the product working as designed.

Personalize the rules, not just the output. “Contains allergens” is useless to a person who is allergic to one thing. Both apps match against the person’s own profile — the FDA Big 9 plus whatever else that person reacts to, or their specific low-FODMAP plan and diet goals — so the answer is about them. Personalization here is not a nicety; it is the difference between information and a decision.

We spent more of the build on when to withhold an answer and how to display the evidence than on raw recognition. That is the opposite of how most AI projects allocate effort, and it is why families use these apps for their children.

The same principles apply well beyond food. Any system that reads a document, a form, an invoice or an inspection photo and makes a call that a person will act on should show its evidence, personalize its rules, and have a well-designed way of saying “not sure”. If you are evaluating such a system, ask to see what it does when it is uncertain. That is where you will find out whether it was built for a demo or for use.