How we test that Darboon hides the right replies
White paper · October 2026
A moderation tool acting for one person makes two decisions about every reply: is it abusive, and is it aimed at the person it protects? Most public test data answers only the first. This paper describes how we measure both before every release, and what we found.
Two questions, not one
The same insult can be written to the author of a post, to another person in the replies, or about someone the post talks about. By default Darboon hides abuse aimed at the author, and at people the author follows when they are attacked in the replies; two repliers trading insults are left alone. Authors who want every abusive reply gone, whoever it targets, can choose that instead. Either way, a tool that recognises abuse but misjudges its target makes mistakes in both directions: it hides people arguing in rough words, or it lets through abuse that reads like criticism.
How we measure
- Controlled datasets. Thousands of real comments, each labelled by people as abusive or not. Each comment is assigned once, by its own fingerprint, to a part used for design or to a test part that no change is ever tuned on.
- The same words, different targets. Each test comment is placed in three situations, changing only what is needed and keeping its words and their strength: addressed to the author, about a person the post mentions, and answering another replier. Because only the target changes, any change in our answer comes from the target. Each situation is judged in both modes.
- Real conversations. A public research dataset of online discussions, annotated for whether abuse is aimed at someone taking part in the conversation or at someone outside it.
- Every release. A change is compared on the design parts, measured once on the test parts, and recorded, and it ships only when it does better. Replies customers report become new checks, and are never tuned on.
Results
The current release, on held-out test sets. F1 balances precision (of the replies we hide, how many deserved it) and recall (of the replies that deserved it, how many we hid); 1.0 is perfect.
| What we measure | Result |
|---|---|
| Recognising abusive replies in Persian (every abusive reply counts) | F1 0.88 · precision 0.90 · recall 0.87 |
| Recognising offensive English posts (every abusive reply counts) | F1 0.91 · precision 0.98 · recall 0.85 |
| Abusive replies in the same-words test, in their thread (every abusive reply counts) | F1 0.94 · precision 0.96 · recall 0.93 |
| Abuse aimed at the author, when only that counts | F1 0.90 · precision 0.94 · recall 0.88 |
| Two other repliers trading insults: left alone | 93% |
| Abuse about someone the post mentions: left alone | 95% |
Over eight releases, F1 at recognising abuse in Persian rose from 0.74 to 0.88, and in English posts from 0.69 to 0.91 in the last three. F1 on the same-words test rose from 0.86 to 0.91.
What we learned
- Recognising abuse and finding its target are different skills. A general-purpose language model was good at the first and weaker at the second: it hid about a third of the insults aimed at someone a post mentioned, as if they were aimed at the author. A second, narrow check that asks only whether a reply is aimed at the author removed most of those mistakes, for about three cents per thousand replies.
- A test set must mean what you mean. One public dataset labelled news headlines and political statements as hate, and plain profanity as not. Scores on it said more about its definition than about our system. We now compare a dataset's definition with ours before we trust its numbers.
- A second opinion helps only where it is good. A second, independent check catches abuse the first one lets through, but in some languages it also adds mistakes. We use it only for the languages where our tests show it helps.
- Precision comes first. On real English conversations, about nine in ten of the replies we hide deserve it. The latest release catches far more abuse in English posts than the one before (recall 0.53 to 0.85) without hiding more by mistake; long, indirect abuse in extended discussions is still harder, and is what we are working on next.
Limits
- Controlled tests measure clear cases; real threads are messier. Every hide and mute can be undone in one click.
- Our deepest test coverage today is Persian and English.
- Pictures are judged separately, and this paper does not report on them yet. Nor does it cover themes.
Questions about how we test: support@darboon.com.