All learning paths Learn See Try Make Sort failures by how bad they are What this lesson is about Put failures in order by the harm they can cause, and defend that order to somebody who disagrees.
Your health bot is rude, gets bus times wrong, and once told a pregnant woman to skip iron tablets. Which do you fix first?
A Ask which fault embarrasses the office most, and put that one at the top. B Fix the bus time first, because it is the quickest of the three to correct. C Order failures by the harm they can cause, not by how often they happen.
Check my recall Why this matters In one published safety study, reviewers tagged each attack with up to two topics from a list of twenty. The list came from earlier published work, from industry moderation rules, and from reading the hundred worst conversations by hand. What made a case serious was the harm it could do, not how odd it looked.
Put failures in order of the harm they can cause, and defend that order to somebody who differs.
In one published safety study, reviewers tagged each attack with up to two topics from a list of twenty. The list came from earlier published work, from industry moderation rules, and from reading the hundred worst conversations by hand. What made a case serious was the harm it could do, not how odd it looked.
Order failures by the harm they can cause, not by how often they happen. Write the reason beside each place, so the order survives a disagreement. Name the failures you could not place instead of forcing them into the list. Task A district health bot is sometimes rude and sometimes gets a bus timing wrong. Once it told a pregnant woman to skip her iron tablets.
Weak approach Counts a month of complaints: forty about rudeness, six about wrong bus times, one about the tablets. Puts rudeness at the top because that column is the longest.
Stronger approach Puts the iron tablet line at the top, with the reason beside it. A woman could stop taking them and come to harm. The rude replies go last.
Why the stronger approach works The strong order can be argued for, because a reason sits beside each place. Counting complaints puts the loudest failure on top, not the worst one.
Try a changed situation Your family argues about which repair in a rented house should be done first. Which rule still applies?
Make something yourself List five failures you have seen in one AI tool, in order of the harm they can cause. Write one line of reason beside each, and name the one you could not place.
Lesson 7 of 8 on Expert, safety & agents next Run a review round with others Part 2 ends in a work sample you can send
Read the primary or official source: Ganguli et al., Red Teaming Language Models to Reduce Harms (arXiv)