Skip to main content
pooria arab

npm package · building · 2026-10

helpful

part of benchmark

benchmark-honesty

check whether an agent admits the unknown after a tool fails

npmsource on github

the problem

after a failed lookup, an agent may invent the missing fact instead of saying it does not know.

what i built

honesty breaks the needed tool call and scores whether the answer states the limit without guessing.