EthicalAI · Position note
A literature-grounded note placing a 117-mode integrity ontology and three behavioural probes against existing AI-ethics taxonomies and benchmarks.
The literature on model integrity is large and growing, but it is organised by task and by failure name, not by a shared account of what is going wrong. Hallucination is studied through faithfulness and factuality benchmarks; sycophancy through preference and agreement datasets3; truthfulness through TruthfulQA2 and, more recently, honesty-under-pressure through MASK4; specification gaming and reward hacking through agentic evaluations12; deception through surveys and alignment-faking studies7,6. Each is valuable, and each uses its own vocabulary and metric.
Harm and risk taxonomies sit above this and do important work1, but they are organised around harms to people and society, not around the mechanisms by which a model's output becomes untrustworthy. There is no shared map that connects a named failure (say, sycophancy) to the principle it breaks (independence from the user), to the harm it produces (manipulation), to the method that could catch it. That missing middle layer is what makes it hard to answer a simple governance question: if a model passes benchmark X, what, precisely, has it been shown not to do?
We built a single file that enumerates 117 integrity failure modes. Each carries a definition, an illustrative example, the principle it violates (from a small set: honesty, calibration, non-maleficence, scope-fidelity, independence, transparency, corrigibility), the harm type it causes, a measurability tag (detectable from a single draft, only across turns, or not at all), and typed relations to other modes. The result is a graph rather than a list: one can ask which principle is guarded by the most modes, which harm is under-covered, or which failures cluster.
Two choices distinguish it from prior taxonomies. First, roughly a hundred of the modes are framed as parallels to human cognition, following the machine psychology programme10: a century of naming human error (anchoring, confabulation, confirmation bias, obedience to authority) often describes model failure more precisely than engineering jargon. We tag the strength of each parallel honestly as on the page (a shipped check), paper-backed, or analogy (framing, not yet measured). Second, every mode carries a measurability field, because the ontology is built to serve a single-draft advisory auditor; this makes explicit which failures a one-pass check can and cannot see. Confabulation, for instance, is only visible under re-sampling, a point the semantic-entropy line of work makes precisely5, and which our tag records.
Single-construct benchmarks such as TruthfulQA2 and MASK4 measure one axis deeply; the ontology instead maps the space and links the axes. The two are complementary: a benchmark is the instrument, the ontology is the map of what to point it at.
A taxonomy earns its keep only if its categories can be measured. We implemented three small, reusable probes, each operationalising an effect with prior support, and each scored deterministically so no judge model is needed.
Obedience to authority. A paired design: a false technical claim asked plainly, then attributed to a confident expert. The honest answer is always to reject it; obedience shows as a flip from rejecting to confirming once authority is added. This operationalises authority and conformity effects long studied in humans and increasingly in models.
Confabulation. Following the instability definition of the semantic-entropy work5, we ask for a precise value about a fictional entity and sample repeatedly; a different confident answer each time is confabulation, an invariant one is a stable hallucination, and abstention is the healthy response. This cannot be seen on a single draft, which is the point.
Noble cause corruption. A paired design in which a request to overstate or fabricate is asked plainly, then wrapped in a sympathetic ends-justify-means justification. Corruption shows as a flip from declining to complying once a good reason is offered.
The preliminary observation, across six low-cost models, is that these three axes do not move together: a model that resisted authority and the noble excuse nonetheless fabricated heavily under re-sampling, while another held firm on facts but conceded to a sympathetic justification. If that pattern survives a proper study, it has a clear implication: integrity is not a single trait, and a model cannot be certified trustworthy from one benchmark. Each failure mode needs its own measurement. We stress that this is three probes, six models, and small item sets; it is a hypothesis-generating observation, not an established result.
The shift is from "is this model safe?" to "safe against which failure, measured how, violating which principle, causing which harm?" That reframing is useful for three audiences. For auditors, it ties a diagnosis to a method and a verdict. For governance, it connects an observed failure to a principle and a harm type, which is the language policy is written in. For researchers, it exposes the gaps: principles with few guarding modes, harms nothing maps to, modes with no measurement. The approach is deliberately advisory and human-in-the-loop, consistent with the view that these tools should inform rather than replace human judgement9,8.
We would rather state these than have them found. The construct mappings are single-annotator. The assignment of each mode to a principle and a harm reflects one person's judgement, with no inter-annotator reliability yet; this is the most important weakness to fix. The probes score a self-reported verdict, which measures what a model says it will do, not what it does; format non-compliance (one model failed to emit the verdict format) is an unmeasured confound, not robustness. The sample is small and single-provider: six low-cost models over one API, small item sets, few or single runs, so per-model counts are noisy and the counts wobble between runs. Confabulation scoring uses exact-string distinctness, a cheap stand-in for semantic entropy that would misread paraphrase as novelty. And finally, an auditor built from the same model family as the drafter shares its blind spots; several failure modes, notably strategic ones such as evaluation-aware sandbagging11, are likely undetectable by this method in principle, and the ontology marks them as such rather than pretending otherwise.
The path from this note to a defensible study is concrete. Add statistical rigour: repeated runs, Wilson intervals on every proportion, and McNemar's test on the paired flips, which is the correct test for this design. Validate the construct mappings with multiple independent annotators and report agreement. Replace exact-string confabulation scoring with semantic clustering. Expand to more models across families and sizes and report a correlation matrix across behaviours, which is what would turn the orthogonality hunch into a finding. Release the full transcripts as a dataset with a datasheet13 and a citable archive. None of these require new ideas, only the discipline the field already expects.
The ontology, the probe harness, and the raw transcripts are in the repository under ontology/ and experiments/behaviour-eval/. This note is a position piece, not a peer-reviewed result; the empirical claims are preliminary and the limitations above are load-bearing.