Overview

Bug injection is the practice of deliberately inserting synthetic bugs into real programs or hardware designs in order to evaluate and compare bug-finding tools, most notably fuzzers. Because organic bugs in mature code bases are scarce and hard to collect at scale, injected bugs serve as a controlled, repeatable ground truth for measuring how effectively a fuzzer can explore a design and surface real defects [arxiv:2208.11088]. The technique is used in both software-fuzzing research (e.g., the Rode0day bug-finding competition and the LAVA-M corpus) and in hardware-fuzzing research on RTL CPU designs [ENCARSIA:sec25].

Software Bug Injection

In the software-fuzzing setting, bug-injection systems add synthetic bugs into real programs to act as proxies for organic defects [arxiv:2208.11088]. A large-scale study consuming over 80 years of CPU time across eight fuzzers and 20 targets (drawn from the Rode0day bug-finding competition and the LAVA-M corpus) compared fuzzer behaviour on synthetic versus organic bugs. Key findings were:

• Synthetic and organic bugs live in dramatically different parts of the program with respect to the "main path" explored by a fuzzer, so a fuzzer's effectiveness on synthetic corpora does not necessarily transfer to real bugs [arxiv:2208.11088].
• No fuzzer found any of the 50 organic CVEs available in the corpus, even though all of them could be triggered, while the same fuzzers could discover many of the injected bugs [arxiv:2208.11088].
• Updates to bug-injection systems have made synthetic bugs harder to find, but they remain significantly easier to find than organic ones in the same targets [arxiv:2208.11088].

Hardware Bug Injection

Manual vs. Automatic Injection

Historically, hardware researchers relied on manual bug insertion in two distinct contexts [ENCARSIA:sec25]:

1. Ad-hoc demo bugs. A few (typically ≤ 7) hand-crafted bugs inserted into a design to showcase one specific fuzzer's effectiveness. These are rarely reused by other researchers and may be biased toward the showcasing fuzzer.
2. Competition bugs. Several dozen bugs inserted by hand to organise hardware-verification competitions such as HACK@DAC. This is labour-intensive, requires industry partnerships and deep familiarity with the target design, and has therefore been performed almost exclusively on CVA6, with bug-inserted designs generally not disclosed publicly.

Manual insertion must be repeated per design and per bug, motivating automatic bug injection as a scalable alternative [ENCARSIA:sec25].

The Two Structural Classes of Bugs

ENCARSIA, a framework for automatic bug injection into RTL CPU designs, is grounded in a large-scale survey of 177 software-observable bugs across four open-source RISC-V CPUs (Ibex, CVA6, Rocket, BOOM) [ENCARSIA:sec25]. Despite affecting a wide range of components and HDL constructs, the bugs in the survey collapse into two recurring syntactic transformation classes:

• Signal mix-ups. Confusion of signals, functions, or other elements in assignments or expressions, usually arising when signals share similar names, types, or purpose. Example (Ibex PR #399): the designer mistakenly used a partial result of an access check (illegal_csr_insn_o) instead of the final result (illegal_csr_priv) in a CSR-write enable expression, allowing unauthorized access to debug CSRs.
• Broken conditionals. Incorrect conditional statements—for example wrong if conditions, missing default cases in case/switch constructs, or signal updates performed in the wrong design state. These arise when designers forget exceptional cases, specify wrong conditional expressions, or make algorithmic mistakes.

Both classes can be expressed as simple circuit transformations on the IR produced by the Yosys synthesis toolchain: signal mix-ups correspond to rewiring expression operands, and broken conditionals correspond to manipulating multiplexer trees that encode conditional logic [ENCARSIA:sec25].

How ENCARSIA Performs Signal Mix-ups

In Yosys's intermediate representation, assignments (called connections) are pairs of wires with their driver signals; for instance assign sig_a = sig_b; is represented as the pair (sig_a, sig_b) [ENCARSIA:sec25]. To inject a mix-up, ENCARSIA replaces the driver signal (right-hand side of an assignment) with another random signal present in the same RTL module. To handle width mismatches, the new driver is sign-extended if it is a constant smaller than the driven wire, consistent with implicit Verilog behaviour [ENCARSIA:sec25].

For logical expressions—translated into a series of interconnected logic cells—mix-ups in expression operands correspond to a wrong signal being connected to a cell port. Because each cell keeps its own internal dictionary mapping ports to connected signals, ENCARSIA inserts an intermediate wire between each cell port and the originally connected signal, so that injecting a mix-up at the intermediate wire is equivalent to injecting a mix-up at the cell port. The intermediate wires are later cleaned up using a standard optimisation pass [ENCARSIA:sec25].

How ENCARSIA Performs Broken-Conditional Transformations

Yosys represents conditionals as multiplexer trees: hierarchical arrangements of multiplexers in which each level narrows the set of expressions that may be assigned to the root signal, with select signals derived from the assignment's conditions [ENCARSIA:sec25]. ENCARSIA abstracts each multiplexer tree into a table that maps specific select-signal combinations to the corresponding chosen expression [ENCARSIA:sec25].

Construction is performed as follows [ENCARSIA:sec25]:

1. ENCARSIA iterates over all wires in the circuit and marks as multiplexer roots those wires that are driven by a multiplexer but do not drive any other multiplexer. These wires correspond to signals that are conditionally assigned with different expressions.
2. Starting at each root, ENCARSIA performs a depth-first search of the multiplexer tree, recording the select-signal values required to follow each path. Every time it reaches an input signal that is not itself a multiplexer output, it discovers a new potential expression for the root, adds a new row to the table with that signal and the recorded select values, expanding the table's columns as necessary and filling existing rows with X (don't-care) for the new columns.
3. The resulting table contains all conditionally chosen expressions for the root with their specific select-signal combinations.

Table transformations model realistic subtle designer mistakes [ENCARSIA:sec25]:

• Removing a row models a designer forgetting to consider a special case.
• Turning a select signal into an X (don't-care) models insufficient constraints on a case.
• Turning a don't-care select signal into a specific value models constraints being placed on cases mistakenly.

These transformations cover the entirety of the bugs in conditional logic revealed in the survey [ENCARSIA:sec25]. After manipulation, each table is translated back into standard circuitry by emitting a Yosys internal pmux cell (a many-input multiplexer with one-hot select encoding). For each row, an eq cell compares the table's stored select combination against the actual select-signal value, and the corresponding entry's expression is connected to one of the pmux inputs. The Yosys backend then translates the pmux cell back into a process resembling the original conditional statement [ENCARSIA:sec25].

Architectural-Observability Check

Not every syntactic transformation produces a bug that is reachable or visible at the architectural level. For example, a transformation that corrupts a triple-modular-redundancy majority-voter to select the minority value instead is undetectable by current CPU fuzzers, because they do not model single-event upsets, so redundant circuits always produce identical outputs and reach the correct consensus [ENCARSIA:sec25]. Including such transformations in an evaluation corpus would skew results by making fuzzers search for innocuous changes.

ENCARSIA uses miter circuits to address this: the original and transformed Design Under Test (DUT) receive identical inputs, and the framework monitors for differences in observable outputs such as the architectural registers. Since the two DUT versions differ only in the transformation performed by ENCARSIA, any difference in observable elements proves that the transformation has an observable effect. Discrepancies are expressed using SystemVerilog Assertions (SVAs) [ENCARSIA:sec25].

To make verification tractable, the formal setup resembles fuzzer initialisation: ENCARSIA resets the CPUs and configures critical control and status registers (for example setting the FS field of mstatus to 1 to enable floating-point instructions), models these configurations with initial-value abstractions and constraints for the first clock cycle, and uses SVA assume statements to restrict the considered input behaviour (e.g., limiting Ibex to a single hardware thread) [ENCARSIA:sec25].

Jasper is then instructed to cover the c_propagated property, which proves that the transformation can provoke an architecturally observable deviation from the original design; if the property is not covered, the transformation is excluded from the evaluation [ENCARSIA:sec25].

ENCARSIA Injection Pipeline (Summary)

Putting the pieces together [ENCARSIA:sec25]:

1. Transformation. Starting from an RTL design (Verilog or Chisel), ENCARSIA compiles it with Yosys and applies either signal-mix-up or broken-conditional transformations to the resulting IR.
2. Architectural-observability check. Each candidate transformation is verified via a miter circuit and SVA-based property proving (using Jasper) to certify that it produces an observable ISA deviation. Injection is limited to one bug per design at a time so that one bug does not hide another.
3. Corpus release. The output is an open synthetic corpus of CPU bugs that can be used as a standardised benchmark for hardware-fuzzer evaluation.

Performance Numbers

ENCARSIA's transformation step is fast: even for a complex design like BOOM, applying a single transformation takes on average less than a second [ENCARSIA:sec25]. Across the three evaluated RISC-V CPUs:

The diversity of transformations is correspondingly wide: ENCARSIA injected 3371 signal mix-ups impacting 2482 signals spread across 121 distinct modules, and 2811 broken conditionals impacting 202 multiplexer trees across 37 distinct modules [ENCARSIA:sec25].

Verification (with Jasper) has substantially lower success rates because not every syntactic transformation yields an architecturally observable bug:

Evaluation with Real Fuzzers

ENCARSIA was evaluated against three state-of-the-art CPU fuzzers—DifuzzRTL, ProcessFuzz (the document uses the spelling "ProcessFuzz"), and Cascade—each of which represents a different approach to CPU fuzzing: DifuzzRTL and ProcessFuzz are guided by distinct coverage metrics, while Cascade is black-box, and they use different algorithms for generating instruction sequences [ENCARSIA:sec25].

To benchmark them, ENCARSIA was used to generate EnCorpus, a unified corpus of 90 bugs for evaluating hardware fuzzers [ENCARSIA:sec25]. On Ibex, Rocket, and BOOM, each fuzzer found roughly 40–42 % of ENCARSIA-injected bugs within 24 hours of fuzzing, exposing shortcomings in coverage guidance and bug-filtering mechanisms [ENCARSIA:sec25]. As a concrete example, Table 7 of the ENCARSIA paper shows Cascade's per-bug detection performance on Ibex: across 15 bugs, Cascade detected only a few mix-ups (bugs 1, 4, and 5) and detected none of the conditional bugs [ENCARSIA:sec25].

Findings on the Quality of Injected Bugs

Across both software and hardware domains, empirical studies surface important caveats about injected bugs:

• Synthetic and organic bugs live in dramatically different parts of the program/design with respect to the "main path" explored by a fuzzer, so a fuzzer's effectiveness on synthetic corpora does not necessarily transfer to real bugs [arxiv:2208.11088].
• In a large software-fuzzing study, no fuzzer found any of the 50 available organic CVEs in the corpus, even though all of them could be triggered; updates to bug-injection systems have made synthetic bugs harder to find, but they remain significantly easier to find than organic ones in the same targets [arxiv:2208.11088].
• In ENCARSIA's evaluation, each of three CPU fuzzers (DifuzzRTL, ProcessFuzz, Cascade) found roughly 40–42 % of ENCARSIA-injected bugs within 24 hours on Ibex, Rocket, and BOOM, exposing shortcomings in coverage guidance and bug-filtering mechanisms [ENCARSIA:sec25].
• Verification (miter + SVA-based property proving) filters out transformations that are not architecturally observable, e.g., changes inside triple-modular-redundancy voters, but at the cost of low success rates for conditional bugs (≈ 2–8 % across Ibex, Rocket, and BOOM) [ENCARSIA:sec25].

Use in LLM-Based Software Engineering Research

Beyond traditional fuzzing, bug injection has also been applied in software-engineering studies of large language models, where deliberate faults are introduced into code samples to assess how consistently and reliably an LLM (e.g., ChatGPT at varying temperature settings) can propose correct fixes [arxiv:2509.06429].

Related Concepts

• Signal mix-up — one of the two structural classes into which injectable CPU bugs fall.
• Broken conditional — the other structural class of injectable CPU bugs.
• Miter circuit — the technique ENCARSIA uses to certify that an injected transformation is architecturally observable.
• EnCorpus — the unified 90-bug corpus generated with ENCARSIA for evaluating hardware fuzzers.