All learning paths Learn See Try Make Chance, and what a model is guessing What this lesson is about See why a system that is right eight times in ten is still wrong twice, and what that costs.
A tool at your office sorts complaints into urgent and ordinary, and gets nine out of ten right. What do you tell the team?
A Report the nine in ten and add that a person should look in now and then. B Wait for the first complaint it gets wrong, then decide whether a person is needed. C A system right eight times in ten is wrong twice, and you cannot say which.
Check my recall Why this matters Google's course shows a model that answers 'no' every time. On data where one case in a hundred is 'yes', it is still 99% right and useless. Being right often and being right where it matters are two different things.
You can say why a system that is usually right still needs a person, and which failures matter.
Google's course shows a model that answers 'no' every time. On data where one case in a hundred is 'yes', it is still 99% right and useless. Being right often and being right where it matters are two different things.
A system right eight times in ten is wrong twice, and you cannot say which. Sort the failures by what each one costs, not by how many there are. 'Likely' is not a measurement until the reply says likely out of what. Task A tool sorts incoming complaints into urgent and ordinary, and is right nine times in ten.
Weak approach Counting a whole week: ninety-four complaints sorted, nine of them wrong, and reporting that number. Which nine went wrong is never opened, so the urgent ones stay hidden.
Stronger approach Reporting that the one in ten it misses are mostly the urgent ones, and those cost the most.
Why the stronger approach works The count was the same in both. Sorting the failures by cost changed what the team had to do.
Try a changed situation A camera at your gate recognises faces and is wrong twice in a hundred. Both errors let in a stranger. What do you report?
Make something yourself Ask an assistant the same factual question ten times, in slightly different words. Record how many answers agreed, and what the two most different ones said.
Lesson 8 of 9 on Reasoning and maths for AI work next Grade a maths answer step by step Part 3 ends in a work sample you can send
Read the primary or official source: Google machine learning crash course · accuracy, precision and recall