Definition

In the fuzzing literature, a bug oracle is a program that determines whether a test case has broken a correctness policy of the program under test (PUT). The policy need not correspond to a software defect in the traditional sense — it can also express other correctness concerns such as performance problems or resource-use violations. The oracle is typically consumed by the InputEval step of a fuzzing loop, which takes the current configuration, a generated test case, and the bug oracle, and produces execution information that may include a detected bug.

Role in a fuzzing loop

Bug oracles sit inside the InputEval stage of a generic fuzzer model. Their verdict feeds into subsequent stages such as ConfUpdate, where the fuzzer decides whether to keep the seed that triggered the oracle, and Continue, which decides whether the campaign should terminate. As a result, the expressiveness and accuracy of the oracle directly shape what classes of defects the fuzzer can discover and how it schedules exploration.

Classes of bug oracles

Exit-code oracles

The simplest and historically earliest class of bug oracle inspects the exit code of the PUT. A non-zero exit, or termination by a fatal signal, is treated as evidence that a bug has been found. This style of oracle has the advantage of requiring no instrumentation of the PUT, but it is limited to exposing major failures: it generally cannot detect subtle defects such as stack-based buffer overflows that do not cause an immediate crash.

Transformation-based oracles

To expose minor bugs (memory safety issues, illegal control flow, undefined behaviour in C, etc.), the PUT can be transformed or instrumented so that additional checks are evaluated at run time. Typical transformations include:

• Spatial memory checks (e.g., out-of-bounds accesses),
• Temporal memory checks (e.g., use-after-free),
• Illegal control-flow detection, and
• Undefined-behaviour checks for C programs.

Each check effectively embeds an oracle inside the binary.

Semantic / differential oracles

When the goal is to expose semantic bugs — incorrect outputs that do not violate memory or control-flow rules — bug oracles commonly rely on differential testing: the behaviour of the PUT is compared against that of a reference model or another similar program. Divergent outputs are taken as strong evidence that the PUT is buggy. This approach is essential when no formal specification or crash signal is available.

Specialized forms

C-syntax bug descriptors (FirmReBugger)

In the firmware-fuzzing benchmark FirmReBugger, bug oracles are expressed as C-syntax expressions of bug descriptors and evaluated by an interpreter. The benchmark framework reports on a fine-grained taxonomy of states — not reached, reached, triggered, and detected — so that monolithic firmware fuzzers can be compared on a realistic, bug-based benchmark (the companion FirmBench ships with 313 software bug oracles). Importantly, this design does not modify the target binary; instead it replays fuzzing seeds through the oracle interpreter, isolating the benchmark from the fuzzer and making it easy to extend with new bug oracles.

Human-in-the-loop oracles (Learn2fix)

When no automated oracle is available, the user reporting the bug can effectively serve as the oracle. The Learn2fix system formalises this setting: it queries the user with alternative test inputs and observed outputs ("When executing this alternative test input, the program produces the following output; is the bug observed?"), and from the labelled responses trains an automatic bug oracle whose accuracy improves as the query budget is spent. This illustrates that the oracle need not be a static program — it can be a learned model that approximates human judgement.

Design trade-offs

The choice of oracle therefore determines both the coverage of bug classes that a fuzzer can find and the engineering cost of running the campaign.

See also

• Differential Testing — the predominant technique for constructing semantic bug oracles.