Coverage Feedback

Definition

Coverage feedback is a fuzzing technique in which signals derived from instrumentation of the target program or hardware design—such as branch conditions, statements, control registers, or multiplexer signals—are reported back to the input generator. These signals guide mutation and generation toward previously unexplored code paths or hardware states [8c432232].

Role in Hardware Fuzzing

Hardware fuzzers harness coverage data, including branch conditions, statements, and multiplexers' control registers or signals, to generate test cases and probe diverse hardware behaviors [8c432232]. Traditional processor fuzzers such as DifuzzRTL and TheHuzz use code coverage and control-register coverage as feedback to guide mutation [5b5279e8]. Hybrid hardware fuzzers such as HyPFuzz and PSOFuzz combine fuzzers with formal tools and optimization algorithms to improve coverage, but they rely on seed generation and mutation engines inherited from traditional processor fuzzers and lack well-defined feedback to determine meaningful instruction sequences that reach deep design regions [5b5279e8].

State-of-the-art hardware fuzzers have demonstrated broader coverage, enhanced scalability, and efficiency in identifying real-world vulnerabilities that have been associated with privilege escalation and arbitrary code execution attacks. However, they often fall short of a 70% coverage threshold in complex hardware such as a RISC-V RocketCore processor [8c432232].

Coverage Feedback in ML-Based and LLM-Based Fuzzers

ChatFuzz

ChatFuzz is an ML-based hardware fuzzer that integrates reinforcement learning guided by code coverage metrics [8c432232]. Its final model results from three consequent training steps [ac90a382]:

1. Initial Training – Unsupervised training based on the GPT-2 model to learn the inner structure of machine language.
2. Model Cleanup Training – PPO-based reinforcement learning with a disassembler as scoring agent to refine the model, cleaning up the learned language and removing bad instruction combinations.
3. Model Optimization – Improving coverage with a PPO-based RL process in which the refined generator is trained through a reward function based on coverage information attained through RTL simulation.

In ChatFuzz's architecture, the RTL Simulator reports coverage information for each test input, which is utilized by the LLM-based Input Generator to optimize the input generation process [ac90a382]. Unlike concurrent work on LLMs in software fuzzing that relies on human-defined prompts, ChatFuzz does not rely on human interaction during training and is additionally steered by coverage metrics [5b5279e8].

On the RISC-V RocketCore, ChatFuzz achieves 75% condition coverage in 52 minutes, in contrast to state-of-the-art fuzzers requiring 30 hours for comparable coverage; it can reach 79.14% condition coverage after approximately 199k test cases. On BOOM, ChatFuzz reaches 97.02% condition coverage in 49 minutes [8c432232].

CovRL-Fuzz

In the software domain, CovRL applies coverage-guided reinforcement learning to LLM-based mutation of JavaScript engines. It integrates coverage feedback directly into the LLM by using TF-IDF to construct a weighted coverage map, which is key in calculating the fuzzing reward applied to the LLM-based mutator through reinforcement learning (public context). CovRL-Fuzz identified 48 real-world security-related bugs in the latest JavaScript engines, including 39 previously unknown vulnerabilities and 11 CVEs (public context).

Database Management Systems (Squirrel)

Coverage feedback has also been applied to database management system (DBMS) fuzzing. Generation-based testing can guarantee the syntactic correctness of inputs but does not utilize feedback such as code coverage to guide path exploration. Squirrel combines language validity with coverage feedback to test DBMSs; in its evaluation it explores 2.0x–10.9x more new edges than mutation-based tools (public context).

Advantages

Coverage feedback offers several benefits over traditional verification methods [8c432232]:

• Broader coverage – coverage-guided exploration reaches deeper regions of the design than random regression or formal verification alone.
• Scalability – coverage feedback automates large-scale exploration of complex designs such as modern processors.
• Efficiency – RL-based reward functions computed from coverage metrics accelerate convergence compared with blind mutation, enabling tools like ChatFuzz to reach high coverage thresholds orders of magnitude faster than state-of-the-art fuzzers.
• Effective vulnerability detection – coverage-guided fuzzers have identified real-world vulnerabilities associated with privilege escalation and arbitrary code execution.

Limitations

Traditional processor fuzzers and their hybrid descendants lack well-defined feedback to determine meaningful instruction sequences that will lead to deep design regions [5b5279e8]. State-of-the-art hardware fuzzers often fall short of a 70% coverage threshold in complex hardware such as a RISC-V RocketCore processor [8c432232]. In the context of LLMs, prior work in software fuzzing relies on human-defined prompts rather than coverage metrics; integrating coverage into LLM training is therefore a non-trivial extension (cf. ChatFuzz) [5b5279e8].

Related Entries

• ChatFuzz – ML-based hardware fuzzer whose PPO-based RL training uses RTL-simulation coverage as the reward signal.
• Beyond Random Inputs: A Novel ML-Based Hardware Fuzzing – the underlying DATE 2024 paper that introduces ChatFuzz and its coverage-feedback-driven training pipeline.