All learning paths Learn See Try Make Audit your own finished batch What this lesson is about Pull a sample from work you have already sent, and count how often you disagree with yourself.
You sent four hundred marked shop photos last Friday. Two free hours sit before the next batch. What do you do with them?
A Pull a sample, decide it again without looking, and count where you differ. B Ask the person who checks your work which photos he sent back last week. C Take forty at random, mark them again with the old mark showing, and count the changes.
Check my recall Why this matters A datasheet is the page that travels with a set of data. It exists so the maker writes down what they checked. Nobody else can measure how steady your own decisions were.
Measure how often you disagree with yourself on work you already sent, and report the figure.
A datasheet is the page that travels with a set of data. It exists so the maker writes down what they checked. Nobody else can measure how steady your own decisions were.
Pull a sample, decide it again without looking, and count where you differ. Disagreeing with yourself is normal and countable; not counting it is the failure. Report the figure with the batch instead of waiting to be asked. Task You sent four hundred labelled shop photos last Friday. You have two free hours before the next batch.
Weak approach Opens the four hundred again, reads the old mark beside each photo, and finds nothing that looks wrong. Reading a decision you already made agrees with it almost every time.
Stronger approach Pulls forty out of the four hundred without looking at the names, and covers the earlier mark. Judges each one afresh: nine come out different, and seven of those are shop boards at night.
Why the stronger approach works Reading your old labels agrees with them. Deciding again with the answer hidden is the only way to catch yourself, and it gives a number.
Try a changed situation A tailor checks his own measurements again before cutting cloth for a wedding order. Which rule still applies?
Make something yourself Take a batch you have already finished. Decide a sample of it again without looking at your old answers. Then write the sample size, how many you changed, and the pattern.
Lesson 6 of 7 on Working well on data projects next Hand a batch over with its notes Part 2 ends in a work sample you can send
Read the primary or official source: Gebru et al., Datasheets for Datasets (arXiv)